devfon b9c6bea75e Add fuse_unk option to SentencePieceBPETokenizer (#574) ...

* Add fuse_unk option to SentencePieceBPETokenizer

* Fix style

Co-authored-by: Anthony MOI <m.anthony.moi@gmail.com>

2021-01-12 16:07:59 -05:00

..

decoders

Python - Improve documentation for decoders and remove useless kwargs

2020-11-23 11:52:51 -05:00

implementations

Add fuse_unk option to SentencePieceBPETokenizer (#574)

2021-01-12 16:07:59 -05:00

models

Python - Improve documentation for models

2020-11-23 11:52:51 -05:00

normalizers

Python - Improve normalizers docs

2020-11-23 11:52:51 -05:00

pre_tokenizers

Python - Update CHANGELOG and stub files

2020-11-27 17:35:34 -05:00

processors

Python - Improve documentation for post-processors

2020-11-23 11:52:51 -05:00

tools

Add a visualization utility to render tokens and annotations in a notebook (#508)

2020-12-04 10:25:56 -05:00

trainers

Python - Improve documentation for trainers

2020-11-23 11:52:51 -05:00

__init__.py

Python - Update CHANGELOG and bump for release

2020-12-08 13:29:35 -05:00

__init__.pyi

Improve docs and fix tests around training

2020-11-28 12:29:35 -05:00