This website requires JavaScript.
Explore
Help
Sign In
mii
/
tokenizers
Watch
1
Star
0
Fork
0
You've already forked tokenizers
mirror of
https://github.com/mii443/tokenizers.git
synced
2025-08-22 16:25:30 +00:00
Code
Issues
Packages
Projects
Releases
Wiki
Activity
Files
9521603e089f2f3735cd1fe4ef7e6f8622d1d775
tokenizers
/
bindings
/
python
/
tests
History
Anthony MOI
9521603e08
Doc - Update Decoder part of the Pipeline page
2020-11-02 17:07:27 -05:00
..
bindings
Fix UnigramTrainer
2020-10-26 16:31:58 -04:00
documentation
Doc - Update Decoder part of the Pipeline page
2020-11-02 17:07:27 -05:00
implementations
Doc - Improve snippets testing
2020-11-02 17:07:27 -05:00
__init__.py
Python - Add some tests utils
2020-04-01 17:25:55 -04:00
test_serialization.py
Fixing a bug where long tokenizer files would be incorrectly deserialized (
#459
)
2020-10-13 18:44:24 +02:00
utils.py
Doc - Quicktour uses python tested code
2020-11-02 17:07:27 -05:00