fbca797b3d
Fixing Trainer with u8 instead of chars. ( #452 )
...
* Fixing Trainer with u8 instead of chars.
Now check both optimized and unoptimized encodings schemes for Unigram.
* Small fixes.
* Fixing makefile.
2020-10-09 18:57:14 +02:00
816632c9fa
Removing --release compat test.
...
- Leaving the one that checks that sampling follows the expected
distribution.
- Marking the python Unigram.train(..) test as slow
- The python Unigram.train(..) test now uses `big.txt` file.
2020-09-02 13:38:14 -04:00
d0366529b7
Use a smaller train file.
2020-09-02 13:38:14 -04:00
7b5c2b92c6
Fixing test dependency.
2020-09-02 13:38:14 -04:00
aa3b39f692
Python - Tests for parallelism with multiprocessing
...
Co-authored-by: Evan Pete Walsh <epwalsh10@gmail.com >
2020-06-23 11:25:39 -04:00
7fd7dfd113
Python - Test CharBPETokenizer
2020-04-01 17:25:56 -04:00
dbc23e20a9
Python - Test Models
2020-04-01 17:25:55 -04:00
023566fbbb
Python - Add some tests utils
2020-04-01 17:25:55 -04:00
