mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-23 00:35:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update CHANGELOG.

Browse Source
This commit is contained in:
Nicolas Patry
2022-01-17 09:56:56 +01:00
parent ab9a2f3100
commit a4cf53f6a7
2 changed files with 18 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
bindings/python/CHANGELOG.md
View File

@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.11.3]
- [#882] Fixing Punctuation deserialize without argument.
- [#868] Fixing missing direction in TruncationParams
- [#860] Adding TruncationSide to TruncationParams
## [0.11.0]
### Fixed
@ -341,6 +347,9 @@ delimiter (Works like `.split(delimiter)`)
- Fix a bug that was causing crashes in Python 3.5
[#882]: https://github.com/huggingface/tokenizers/pull/850
[#868]: https://github.com/huggingface/tokenizers/pull/850
[#860]: https://github.com/huggingface/tokenizers/pull/850
[#850]: https://github.com/huggingface/tokenizers/pull/850
[#844]: https://github.com/huggingface/tokenizers/pull/844
[#845]: https://github.com/huggingface/tokenizers/pull/845

9
tokenizers/CHANGELOG.md
View File

@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.11.1]
- [#882] Fixing Punctuation deserialize without argument.
- [#868] Fixing missing direction in TruncationParams
- [#860] Adding TruncationSide to TruncationParams
## [0.11.0]
### Fixed
@ -125,6 +131,9 @@ advised, but that's not the question)
split up in multiple bytes
- [#174]: The `LongestFirst` truncation strategy had a bug
[#882]: https://github.com/huggingface/tokenizers/pull/850
[#868]: https://github.com/huggingface/tokenizers/pull/850
[#860]: https://github.com/huggingface/tokenizers/pull/850
[#403]: https://github.com/huggingface/tokenizers/pull/403
[#377]: https://github.com/huggingface/tokenizers/pull/377
[#355]: https://github.com/huggingface/tokenizers/pull/355