d2fc0e4836
Doc - Update API Reference for Encoding
2020-11-02 17:07:27 -05:00
940f8bd8fa
Update PyO3 ( #426 )
2020-09-22 12:00:20 -04:00
52082b5476
New clippy comments?
2020-09-02 16:32:50 +02:00
83a52c8080
Replace Model and Trainer Containers.
...
* Implement changes necessary from generic Model in Tokenizer.
* Temporarily disable training in Python since Clone can't be
derived for Model until all components have been replaced.
* Prefix Python types in Rust with Py.
2020-08-04 15:59:33 -04:00
7833965dc4
Update Python bindings with new interface
2020-08-03 16:18:59 -04:00
de9feae0b5
Python - Make Encoding pickable
2020-05-27 13:46:37 -04:00
6a70162d78
Python - Make all relevant classes pickable
2020-05-27 13:46:37 -04:00
7d2b59b0aa
Rust - Add len() and is_empty() on Encoding
2020-04-24 11:44:10 -04:00
873ac2d9a8
Python - Add missing char_to_word
2020-04-16 18:20:30 -04:00
c5e22c14cb
Python - Improve mappings on Encoding
2020-04-16 14:23:37 -04:00
3ad1360210
Word indices are None for special tokens
2020-04-09 09:52:02 -04:00
2dc48e56ac
Python - Update pyo3 version
...
* Use __new__ instead of static method as model constructors
2020-04-06 21:20:16 +02:00
4341c79d85
Python - last fixes on Encoding bindings/typings
2020-03-26 15:42:45 -04:00
9bd9e0b3c1
Expose post_process on the Tokenizer
2020-03-26 15:42:45 -04:00
9ce895550b
Add some new merging capability on Encoding
2020-03-26 15:42:44 -04:00
1150751ab6
Python - Update mappings API
2020-03-26 15:42:44 -04:00
a397a1da63
Python - Expose encode method on Model
2020-03-26 15:42:44 -04:00
8de6ef5a37
Python - Bind new Encoding's mappings
2020-03-26 15:42:44 -04:00
60a4fb35f4
Python - Update bindings
2020-03-16 10:36:42 -04:00
d8a73c89a7
Python - Add Encoding length
2020-02-18 18:24:13 -05:00
3adf199a0c
fix pad calls
2020-02-05 14:49:47 -05:00
8decd020cb
Python - Provide mapping to original offsets
...
As requested on #81
2020-02-05 13:33:19 -05:00
b90104e705
Update Python bindings
2020-02-03 11:38:52 -05:00
05275a9391
python: fix inverted normalized/original string range
2020-01-31 11:09:55 -05:00
1a802cb484
fix typos
2020-01-10 10:47:36 +01:00
d46ea842c2
Python - IndexableString accepts tuples directly
2020-01-10 00:32:30 -05:00
988159a998
Hotfix Python bindings for 32-bit systems
2020-01-08 13:42:35 -05:00
88711d5717
Python - IndexableString in Encoding
2020-01-08 00:06:57 -05:00
dc76e11768
Python - Provide __repr__ for Encoding
2020-01-07 21:33:45 -05:00
0079a7a6b7
Python - Add NormalizedString + doc/typings
2020-01-06 17:55:22 -05:00
4677a09626
Python - Expose pad and truncate on Encoding
2019-12-30 12:56:07 -05:00
06d515d41b
Python - Add ability to retrieve a range of string
2019-12-29 01:37:03 -05:00
049029dc42
Python - Restore methods on Encoding
2019-12-29 01:26:42 -05:00
e01d4f2052
Python - Remove misleading __repr__
2019-12-23 17:27:59 -05:00
6766585965
Python - Do not expose non working features of Encoding
2019-12-17 17:43:42 -05:00
d80f752ec9
Python - Add some missing Encoding bindings
2019-12-16 19:38:18 -05:00
6c294c60b0
Python - Add Encoding repr + improve example
2019-12-10 15:18:07 -05:00
99773d9ce4
Python - Add encoding getters
2019-12-10 15:17:41 -05:00
8cedc5f1f6
Update Python bindings for Encoding
2019-12-10 12:38:36 -05:00
