mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-23 00:35:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Python release CI (#2)

Browse Source
This commit is contained in:
MOI Anthony
2019-12-02 19:04:25 -05:00
committed by GitHub
parent 1a52cda912
commit 5f31ac3f75
5 changed files with 116 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
bindings/python/setup.py
View File

@ -5,6 +5,9 @@ setup(
name="tokenizers",
version="0.0.2",
description="Fast and Customizable Tokenizers",
long_description=open("README.md", "r", encoding="utf-8").read(),
long_description_content_type="text/markdown",
keywords="NLP tokenizer BPE transformer deep learning"
author="Anthony MOI",
author_email="anthony@huggingface.co",
url="https://github.com/huggingface/tokenizers",