mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-22 16:25:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Changelog.

Browse Source
This commit is contained in:
Nicolas Patry
2022-01-17 22:31:54 +01:00
parent 9b85424520
commit a8e07d734f
2 changed files with 16 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
bindings/python/CHANGELOG.md
View File

@ -4,6 +4,10 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.11.4]
- [#884] Fixing bad deserialization following inclusion of a default for Punctuation
## [0.11.3]
- [#882] Fixing Punctuation deserialize without argument.
@ -347,9 +351,10 @@ delimiter (Works like `.split(delimiter)`)
- Fix a bug that was causing crashes in Python 3.5
[#882]: https://github.com/huggingface/tokenizers/pull/850
[#868]: https://github.com/huggingface/tokenizers/pull/850
[#860]: https://github.com/huggingface/tokenizers/pull/850
[#884]: https://github.com/huggingface/tokenizers/pull/884
[#882]: https://github.com/huggingface/tokenizers/pull/882
[#868]: https://github.com/huggingface/tokenizers/pull/868
[#860]: https://github.com/huggingface/tokenizers/pull/860
[#850]: https://github.com/huggingface/tokenizers/pull/850
[#844]: https://github.com/huggingface/tokenizers/pull/844
[#845]: https://github.com/huggingface/tokenizers/pull/845

11
tokenizers/CHANGELOG.md
View File

@ -4,6 +4,10 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.11.2]
- [#884] Fixing bad deserialization following inclusion of a default for Punctuation
## [0.11.1]
- [#882] Fixing Punctuation deserialize without argument.
@ -131,9 +135,10 @@ advised, but that's not the question)
split up in multiple bytes
- [#174]: The `LongestFirst` truncation strategy had a bug
[#882]: https://github.com/huggingface/tokenizers/pull/850
[#868]: https://github.com/huggingface/tokenizers/pull/850
[#860]: https://github.com/huggingface/tokenizers/pull/850
[#884]: https://github.com/huggingface/tokenizers/pull/884
[#882]: https://github.com/huggingface/tokenizers/pull/882
[#868]: https://github.com/huggingface/tokenizers/pull/868
[#860]: https://github.com/huggingface/tokenizers/pull/860
[#403]: https://github.com/huggingface/tokenizers/pull/403
[#377]: https://github.com/huggingface/tokenizers/pull/377
[#355]: https://github.com/huggingface/tokenizers/pull/355