Logo
Explore Help
Sign In
mii/tokenizers
1
0
Fork 0
You've already forked tokenizers
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-24 00:59:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
525 Commits 1 Branch 0 Tags
505c428f7267667859bf8ffd7485db3269488e3b
Commit Graph

12 Commits

Author SHA1 Message Date
Anthony MOI
505c428f72 Python - Update example.py with new API 2020-02-10 11:55:14 -05:00
Morgan Funtowicz
374f944e32 Use the same vocabs/merges for Python and Rust comparison.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
2020-01-15 11:57:34 +01:00
Anthony MOI
cc33418044 Python - Update examples with getter/setter 2020-01-07 15:23:11 -05:00
Anthony MOI
7eebd06409 Python - Improve imports 2020-01-06 12:03:01 -05:00
Anthony MOI
3779bf3e19 Python - Update example 2019-12-29 00:38:37 -05:00
Anthony MOI
a7734ffc9f Python - Update doc and readme for add_prefix_space 2019-12-26 10:34:53 -05:00
Anthony MOI
4bc5a7bbe7 Python - fix example 2019-12-24 11:20:40 -05:00
Anthony MOI
036ee603f4 Python - Update example 2019-12-16 18:50:21 -05:00
Anthony MOI
e93cc62a71 Python - Handle kwargs for bert modules 2019-12-13 15:28:29 -05:00
Anthony MOI
6c294c60b0 Python - Add Encoding repr + improve example 2019-12-10 15:18:07 -05:00
Anthony MOI
018f57f054 Python - Update example 2019-12-09 12:51:05 -05:00
Anthony MOI
6437c40235 Python - PoC Custom PreTokenizer 2019-11-24 00:52:13 -05:00
Powered by Gitea Version: 1.24.2 Page: 122ms Template: 14ms
English
Bahasa Indonesia Deutsch English Español Français Gaeilge Italiano Latviešu Magyar nyelv Nederlands Polski Português de Portugal Português do Brasil Suomi Svenska Türkçe Čeština Ελληνικά Български Русский Українська فارسی മലയാളം 日本語 简体中文 繁體中文(台灣) 繁體中文(香港) 한국어
Licenses API