mirror of
https://github.com/mii443/tokenizers.git
synced 2025-08-22 16:25:30 +00:00
Update info on environment variable for threading (#1150)
* Update env var name for threading * Update env var name for threading
This commit is contained in:
@ -128,8 +128,8 @@ fn main() -> Result<()> {
|
|||||||
## Additional information
|
## Additional information
|
||||||
|
|
||||||
- tokenizers is designed to leverage CPU parallelism when possible. The level of parallelism is determined
|
- tokenizers is designed to leverage CPU parallelism when possible. The level of parallelism is determined
|
||||||
by the total number of core/threads your CPU provides but this can be tuned by setting the `RAYON_RS_NUM_CPUS`
|
by the total number of core/threads your CPU provides but this can be tuned by setting the `RAYON_RS_NUM_THREADS`
|
||||||
environment variable. As an example setting `RAYON_RS_NUM_CPUS=4` will allocate a maximum of 4 threads.
|
environment variable. As an example setting `RAYON_RS_NUM_THREADS=4` will allocate a maximum of 4 threads.
|
||||||
**_Please note this behavior may evolve in the future_**
|
**_Please note this behavior may evolve in the future_**
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
@ -116,8 +116,8 @@
|
|||||||
//! # Additional information
|
//! # Additional information
|
||||||
//!
|
//!
|
||||||
//! - tokenizers is designed to leverage CPU parallelism when possible. The level of parallelism is determined
|
//! - tokenizers is designed to leverage CPU parallelism when possible. The level of parallelism is determined
|
||||||
//! by the total number of core/threads your CPU provides but this can be tuned by setting the `RAYON_RS_NUM_CPUS`
|
//! by the total number of core/threads your CPU provides but this can be tuned by setting the `RAYON_RS_NUM_THREADS`
|
||||||
//! environment variable. As an example setting `RAYON_RS_NUM_CPUS=4` will allocate a maximum of 4 threads.
|
//! environment variable. As an example setting `RAYON_RS_NUM_THREADS=4` will allocate a maximum of 4 threads.
|
||||||
//! **_Please note this behavior may evolve in the future_**
|
//! **_Please note this behavior may evolve in the future_**
|
||||||
//!
|
//!
|
||||||
//! # Features
|
//! # Features
|
||||||
|
Reference in New Issue
Block a user