mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-12-07 13:18:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

Rust & Python - Update CHANGELOGs

Browse Source
This commit is contained in:
Anthony MOI
2020-02-26 10:51:40 -05:00
parent 4023bb202c
commit 3b10d640d5
2 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
bindings/python/CHANGELOG.md
View File

@@ -3,6 +3,7 @@
Fixes:
- Some default tokens were missing from `BertWordPieceTokenizer` (cf [#160](https://github.com/huggingface/tokenizers/issues/160))
# v0.5.2
- Do not open all files directly while training ([#163](https://github.com/huggingface/tokenizers/issues/163))
## Fixes:
- We introduced a bug related to the saving of the WordPiece model in 0.5.2: The `vocab.txt` file was named