Nicolas Patry 792d618006 Adding a new "Replace" normalizer that takes a string and replaces it ...

with another String (for now).

2020-09-18 12:24:39 +02:00

..

decoders

Move Python source to subdirectory.

2020-07-25 23:40:47 +02:00

implementations

Improvements on spm parity: (#401)

2020-09-15 22:21:02 +02:00

models

Address @n1t0 comments.

2020-09-02 16:32:50 +02:00

normalizers

Adding a new "Replace" normalizer that takes a string and replaces it

2020-09-18 12:24:39 +02:00

pre_tokenizers

Adding a new pre_tokenizer: Digits.

2020-09-03 21:03:45 +02:00

processors

TemplateProcessing - Add @narsil suggestions

2020-09-10 15:04:19 -04:00

trainers

Expose the trainer to Python bindings.

2020-09-02 13:38:14 -04:00

__init__.py

Python - Bump version for 0.9.0.dev1 release

2020-09-02 22:31:01 -04:00

__init__.pyi

Remove Tokenizer::normalize

2020-08-19 12:42:12 -04:00