29fef1e7aa
[remove black] And use ruff ( #1436 )
...
* nits
* Fixing deps.
* Ruff update.
* Import order matters.
* Fix.
* Revert ruff fix.
* Visualizer.
* Putting back the imports.
---------
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com >
2024-03-12 11:24:21 +01:00
6113666624
Updating python formatting. ( #1079 )
...
* Updating python formatting.
* Forgot gh action.
* Skipping isort to prevent circular imports.
* Updating stub.
* Removing `isort` (it contradicts `stub.py`).
* Fixing weird stub black/isort disagreeement.
2022-10-05 15:29:33 +02:00
da4c7b10e4
Add a way to specify the unknown token in SentencePieceUnigramTokenizer python implem ( #762 )
...
* add a way to specify the unknown token in `SentencePieceUnigramTokenizer`
* add test that verify that an exception is raised for the missing unknown token
* style
* add test tokens
2021-08-12 09:42:44 -04:00
d94fa220b6
Python - Add train_from_iterator to implementations
2021-01-07 09:02:20 -05:00
