mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-22 16:25:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updating the docs with the new command. (#1333)

Browse Source
This commit is contained in:
Nicolas Patry
2023-08-29 13:15:26 +02:00
committed by GitHub
parent d2010d5165
commit 8e522a38d9
5 changed files with 7 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
bindings/python/README.md
View File

@ -64,8 +64,7 @@ python -m venv .env
source .env/bin/activate
# Install `tokenizers` in the current virtual env
pip install setuptools_rust
python setup.py install
pip install -e .
```
### Load a pretrained tokenizer from the Hub