Logo
Explore Help
Sign In
mii/tokenizers
1
0
Fork 0
You've already forked tokenizers
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-24 00:59:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
405 Commits 1 Branch 0 Tags
374f944e3217eadb4e83a5a239ce23c090404c66
Commit Graph

11 Commits

Author SHA1 Message Date
Morgan Funtowicz
374f944e32 Use the same vocabs/merges for Python and Rust comparison.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
2020-01-15 11:57:34 +01:00
Anthony MOI
cc33418044 Python - Update examples with getter/setter 2020-01-07 15:23:11 -05:00
Anthony MOI
7eebd06409 Python - Improve imports 2020-01-06 12:03:01 -05:00
Anthony MOI
3779bf3e19 Python - Update example 2019-12-29 00:38:37 -05:00
Anthony MOI
a7734ffc9f Python - Update doc and readme for add_prefix_space 2019-12-26 10:34:53 -05:00
Anthony MOI
4bc5a7bbe7 Python - fix example 2019-12-24 11:20:40 -05:00
Anthony MOI
036ee603f4 Python - Update example 2019-12-16 18:50:21 -05:00
Anthony MOI
e93cc62a71 Python - Handle kwargs for bert modules 2019-12-13 15:28:29 -05:00
Anthony MOI
6c294c60b0 Python - Add Encoding repr + improve example 2019-12-10 15:18:07 -05:00
Anthony MOI
018f57f054 Python - Update example 2019-12-09 12:51:05 -05:00
Anthony MOI
6437c40235 Python - PoC Custom PreTokenizer 2019-11-24 00:52:13 -05:00
Powered by Gitea Version: 1.24.2 Page: 85ms Template: 12ms
English
Bahasa Indonesia Deutsch English Español Français Gaeilge Italiano Latviešu Magyar nyelv Nederlands Polski Português de Portugal Português do Brasil Suomi Svenska Türkçe Čeština Ελληνικά Български Русский Українська فارسی മലയാളം 日本語 简体中文 繁體中文(台灣) 繁體中文(香港) 한국어
Licenses API