mirror of
https://github.com/mii443/tokenizers.git
synced 2025-08-22 16:25:30 +00:00
9 lines
275 B
PHP
9 lines
275 B
PHP
Crates.io
|
|
----------------------------------------------------------------------------------------------------
|
|
|
|
🤗 Tokenizers is available on `crates.io <https://crates.io/crates/tokenizers>`__.
|
|
|
|
You just need to add it to your :obj:`Cargo.toml`::
|
|
|
|
tokenizers = "0.10"
|