Sign in Sign up
cargo

snaptokens

Fast BPE tokenization from local Hugging Face tokenizer.json files

cargo View on Cargo

Activity

Latest release
6d ago
Total releases
6
Cadence
~daily
Last 12 months
6

Reach

Downloads
279

Details

License
Apache-2.0
First release
Jul 21, 2026
Releases
Version Released
0.2.6 patch
0.2.5 patch
0.2.4 patch
0.2.3 patch
0.2.2 patch
0.2.1 initial