mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-22 16:25:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Making Tokenizer clone. (#1152)

Browse Source
This commit is contained in:
Nicolas Patry
2023-01-23 10:12:35 +01:00
committed by GitHub
parent 1fcd90b0b7
commit b861d48b06
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tokenizers/src/tokenizer/mod.rs
View File

@ -399,7 +399,7 @@ where
}
}
#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Tokenizer(
TokenizerImpl<
ModelWrapper,