mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-12-09 06:08:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
498 Commits 1 Branch 0 Tags
a1284f6220a02a2cba5f2549bba3db6deb1ea0d7
Commit Graph

1 Commits

Author SHA1 Message Date
Morgan Funtowicz
be10f542ce Added SentencePiece and YouTokenToMe model extractors. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-01-08 22:55:00 +01:00