mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-09-01 14:59:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
06025e4ca151dcc7f6a4872e8857e8f175e8d3ac
tokenizers/bindings/python/scripts
History
Lysandre Debut 4b0dc6b947 Fix SPM conversions (#686) 
* Fix SPM conversions

* Update changelog

Co-authored-by: Anthony MOI <m.anthony.moi@gmail.com>
2021-05-20 09:55:55 -04:00
..
convert.py
Fix SPM conversions (#686)
2021-05-20 09:55:55 -04:00
sentencepiece_extractor.py
Black pass.
2020-08-31 14:05:39 -04:00
spm_parity_check.py
Fix SentencePiece tokenizers conversion
2021-02-03 12:44:46 -05:00