Logo
Explore Help
Sign In
mii/tokenizers
1
0
Fork 0
You've already forked tokenizers
mirror of https://github.com/mii443/tokenizers.git synced 2025-12-09 14:18:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
14e3ab3787ded8c162c464fac73b4055213cf052
tokenizers/bindings/python/tokenizers
History
Anthony MOI 14e3ab3787 Python - fix style
2020-03-26 15:42:45 -04:00
..
decoders
Python - Bindings for Wordpiece decoder's cleanup
2020-02-13 17:50:37 -05:00
implementations
Forward type_id in encode_tokenized/encode_tokenized_batch python binding.
2020-03-26 15:42:45 -04:00
models
TokenizedSequence / TokenizedSequenceWithOffsets needs to be declared in .py files not only .pyi
2020-03-26 15:42:45 -04:00
normalizers
Python - Update bindings
2020-03-09 18:37:03 -04:00
pre_tokenizers
Python - Update bindings
2020-03-09 18:37:03 -04:00
processors
Keep default values as true
2020-03-10 12:58:53 -04:00
trainers
Python - Add bindings for new AddedToken options
2020-03-24 20:58:45 -04:00
__init__.py
Python - fix style
2020-03-26 15:42:45 -04:00
__init__.pyi
Expose post_process on the Tokenizer
2020-03-26 15:42:45 -04:00
Powered by Gitea Version: 1.24.5 Page: 124ms Template: 6ms
English
Bahasa Indonesia Deutsch English Español Français Gaeilge Italiano Latviešu Magyar nyelv Nederlands Polski Português de Portugal Português do Brasil Suomi Svenska Türkçe Čeština Ελληνικά Български Русский Українська فارسی മലയാളം 日本語 简体中文 繁體中文(台灣) 繁體中文(香港) 한국어
Licenses API