Nicolas Patry 275ee6d4c4 Making convert script machine agnostic.

2020-09-22 08:21:38 +02:00

..

convert.py

Making convert script machine agnostic.

2020-09-22 08:21:38 +02:00

sentencepiece_extractor.py

Black pass.

2020-08-31 14:05:39 -04:00

spm_parity_check.py

Adding a new convert script, that will convert all python Tokenizer code

2020-09-22 08:21:38 +02:00