mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-22 16:25:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #60 from adelevie/patch-1

Browse Source 
Typo in README.md
This commit is contained in:
MOI Anthony
2020-01-14 09:18:58 -05:00
committed by GitHub
parent b41ce0e9d6 554d6f7fd9
commit 8e2328925b
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -59,7 +59,7 @@ Encoding(num_tokens=13, attributes=[ids, type_ids, tokens, offsets, attention_ma
'😁'
```
And training an new vocabulary is just as easy:
And training a new vocabulary is just as easy:
```python
# You can also train a BPE/Byte-levelBPE/WordPiece vocabulary on your own files