mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-23 16:49:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
2b72017e17c36a7f90a3301997614f2e2818a511
tokenizers/bindings/python/py_src/tokenizers/normalizers
History
Nicolas Patry d2c8190a0f Creating normalizers.Prepend (To be used instead of Metaspace). (#1194) 
* Creating `normalizers.Prepend` (To be used instead of `Metaspace`).

* Linting + stub.

* Fixing pickling/unpickling by setting a default.

* Black.
2023-03-24 00:33:31 +01:00
..
__init__.py
Creating normalizers.Prepend (To be used instead of Metaspace). (#1194)
2023-03-24 00:33:31 +01:00
__init__.pyi
Creating normalizers.Prepend (To be used instead of Metaspace). (#1194)
2023-03-24 00:33:31 +01:00