diff --git a/tokenizers/README.md b/tokenizers/README.md index 0c9266d2..ab558c96 100644 --- a/tokenizers/README.md +++ b/tokenizers/README.md @@ -1,4 +1,16 @@ -# Tokenizers +

+
+ +
+

+

+ Build + + GitHub + +

+
+ The core of `tokenizers`, written in Rust. Provides an implementation of today's most used tokenizers, with a focus on performance and