6113666624
Updating python formatting. ( #1079 )
...
* Updating python formatting.
* Forgot gh action.
* Skipping isort to prevent circular imports.
* Updating stub.
* Removing `isort` (it contradicts `stub.py`).
* Fixing weird stub black/isort disagreeement.
2022-10-05 15:29:33 +02:00
c1100ec542
Clippy fixes. ( #846 )
...
* Clippy fixes.
* Drop support for Python 3.6
* Remove other 3.6
* Re-enabling caches for build (5h + seems too long and issue seems
solved)
https://github.com/actions/virtual-environments/issues/572
* `npm audit fix`.
* Fix yaml ?
* Pyarrow issue fixed: https://github.com/huggingface/datasets/pull/2268
* Installing dev libraries.
* Install python dev elsewhere ?
* Typo.
* No sudo.
* ...
* Testing the GH again.
* Maybe v2 will fix ?
* Fixing tests on MacOS Python 3.8+
2021-12-15 15:55:48 +01:00
324aa2930a
Doc - Improve python and node tests
2020-11-02 17:07:27 -05:00
4cf0a0b72c
Doc - Quicktour uses python tested code
2020-11-02 17:07:27 -05:00
88556790e7
Fixing a bug where long tokenizer files would be incorrectly deserialized ( #459 )
...
* Fixing a bug where long tokenizer files would be incorrectly
deserialized
- Add a bunch of tests to check deserialization behaviour
- One tests also confirms current Single deserialization of Sequence.
* Better test locations for Windows + no file dependency in Python binding
Rust side.
* Adressing @n1t0 comments.
2020-10-13 18:44:24 +02:00
fbca797b3d
Fixing Trainer with u8 instead of chars. ( #452 )
...
* Fixing Trainer with u8 instead of chars.
Now check both optimized and unoptimized encodings schemes for Unigram.
* Small fixes.
* Fixing makefile.
2020-10-09 18:57:14 +02:00
816632c9fa
Removing --release compat test.
...
- Leaving the one that checks that sampling follows the expected
distribution.
- Marking the python Unigram.train(..) test as slow
- The python Unigram.train(..) test now uses `big.txt` file.
2020-09-02 13:38:14 -04:00
d0366529b7
Use a smaller train file.
2020-09-02 13:38:14 -04:00
7b5c2b92c6
Fixing test dependency.
2020-09-02 13:38:14 -04:00
aa3b39f692
Python - Tests for parallelism with multiprocessing
...
Co-authored-by: Evan Pete Walsh <epwalsh10@gmail.com >
2020-06-23 11:25:39 -04:00
7fd7dfd113
Python - Test CharBPETokenizer
2020-04-01 17:25:56 -04:00
dbc23e20a9
Python - Test Models
2020-04-01 17:25:55 -04:00
023566fbbb
Python - Add some tests utils
2020-04-01 17:25:55 -04:00
