Morgan Funtowicz 374f944e32 Use the same vocabs/merges for Python and Rust comparison. ...

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

2020-01-15 11:57:34 +01:00

..

custom_pre_tokenizer.py

Python - Update examples with getter/setter

2020-01-07 15:23:11 -05:00

example.py

Use the same vocabs/merges for Python and Rust comparison.

2020-01-15 11:57:34 +01:00

train_bert_wordpiece.py

Updated train_bert_wordpiece.py as well.

2020-01-14 13:32:02 +01:00

train_bytelevel_bpe.py

Updated train_bytelevel_bpe.py to use the high level Python API.

2020-01-14 12:00:50 +01:00