mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-24 00:59:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
ee2af9e99a00aaa552a41be3f547e69018bf948f
tokenizers/bindings/python/scripts
History
Lysandre Debut 4b0dc6b947 Fix SPM conversions (#686) 
* Fix SPM conversions

* Update changelog

Co-authored-by: Anthony MOI <m.anthony.moi@gmail.com>
2021-05-20 09:55:55 -04:00
..
convert.py
Fix SPM conversions (#686)
2021-05-20 09:55:55 -04:00
sentencepiece_extractor.py
Black pass.
2020-08-31 14:05:39 -04:00
spm_parity_check.py
Fix SentencePiece tokenizers conversion
2021-02-03 12:44:46 -05:00