mirror of
https://github.com/mii443/tokenizers.git
synced 2025-12-09 22:28:29 +00:00
504d8c85d8
This is actually a legacy function that doesn't really make sense now, and is getting really difficult to keep. So we remove it.