mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-22 16:25:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Removing python3.6 from manylinux it's not supported anymore. (#1063)

Browse Source
This commit is contained in:
Nicolas Patry
2022-09-19 12:22:02 +02:00
committed by GitHub
parent 7c146d9ce5
commit 655f4057b7
2 changed files with 5 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
bindings/python/build-wheels.sh
View File

@ -4,7 +4,7 @@ set -ex
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"
for PYBIN in /opt/python/{cp36-cp36m,cp37-cp37m,cp38-cp38,cp39-cp39,cp310-cp310}/bin; do
for PYBIN in /opt/python/{cp37-cp37m,cp38-cp38,cp39-cp39,cp310-cp310}/bin; do
export PYTHON_SYS_EXECUTABLE="$PYBIN/python"
"${PYBIN}/pip" install -U setuptools-rust==0.11.3

4
bindings/python/test2.py Normal file
View File

@ -0,0 +1,4 @@
from tokenizers import Tokenizer
tokenizer = Tokenizer.from_file("/home/nicolas/Downloads/tokenizer-wiki.txt")