mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-22 16:25:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
1a84958cc83f0a5ac1be53207ffec160fdd1774b
tokenizers/bindings
History
Nicolas Patry 1a84958cc8 Fixing bad deserialization following inclusion of a default for Punctuation. (#884) 
* Fixing bad deserialization following inclusion of a default for
`Punctuation`.

* don't remove the type now...

* Adding slow test to run on all the tokenizers of the hub.

* `PartialEq` everywhere.

* Forcing `type` to exist on the `pre_tokenizers`.
2022-01-17 22:28:25 +01:00
..
node
Fixing bad deserialization following inclusion of a default for Punctuation. (#884)
2022-01-17 22:28:25 +01:00
python
Fixing bad deserialization following inclusion of a default for Punctuation. (#884)
2022-01-17 22:28:25 +01:00