mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-09-03 15:59:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
b235f8552721d816013206c1a33e8cc37a764a13
tokenizers/bindings/python/scripts
History
Lysandre Debut 4b0dc6b947 Fix SPM conversions (#686) 
* Fix SPM conversions

* Update changelog

Co-authored-by: Anthony MOI <m.anthony.moi@gmail.com>
2021-05-20 09:55:55 -04:00
..
convert.py
Fix SPM conversions (#686)
2021-05-20 09:55:55 -04:00
sentencepiece_extractor.py
Black pass.
2020-08-31 14:05:39 -04:00
spm_parity_check.py
Fix SentencePiece tokenizers conversion
2021-02-03 12:44:46 -05:00