mii/tokenizers
1
0
Fork 0
mirror of https://github.com/mii443/tokenizers.git synced 2025-08-22 16:25:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

CD backports (#1318)

Browse Source 
* CD backports

follow
huggingface/safetensors#317

* fix node bindings?

`cargo check` doesnt work on my local configuration from `tokenizers/bindings/node/native`
i don't think it will be a problem but i have difficulty telling

* backport #315

* safetensors#317 back ports
This commit is contained in:
Chris Ha
2023-08-11 01:52:22 +09:00
committed by GitHub
parent d47d3e377c
commit 862046ac94
2 changed files with 52 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
bindings/node/native/src/tokenizer.rs
View File

@ -701,7 +701,7 @@ declare_types! {
let params_obj = neon_serde::to_value(&mut cx, &TruncationParams(options.clone()))?;
let mut this = cx.this();
let guard = cx.lock();
this.borrow_mut(&guard)
let _ = this.borrow_mut(&guard)
.tokenizer.write().unwrap()
.with_truncation(Some(options));