mirror of
https://github.com/mii443/tokenizers.git
synced 2025-12-05 04:08:22 +00:00
504d8c85d8
This is actually a legacy function that doesn't really make sense now, and is getting really difficult to keep. So we remove it.