mirror of
https://github.com/mii443/tokenizers.git
synced 2025-09-01 23:09:34 +00:00
21 lines
320 B
Markdown
21 lines
320 B
Markdown
### Python Bindings
|
|
|
|
```
|
|
# This expect the rust chain to be nightly
|
|
rustup update
|
|
rustup default nightly
|
|
|
|
# In this folder:
|
|
python3 -m venv .env
|
|
source .env/bin/activate
|
|
pip install maturin
|
|
maturin develop --release
|
|
|
|
# Then test:
|
|
pip install transformers
|
|
|
|
python example.py --file <FILE_PATH>
|
|
# or
|
|
python example.py
|
|
```
|