Sign in Sign up
cargo

dynamo-tokenizers

Standalone HuggingFace, tiktoken, fastokens, and Baseten tokenizer implementations for LLM inference serving.

cargo View on Cargo github View on Github

Activity

Latest release
6h ago
Total releases
28
Cadence
~4 days
Last 12 months
28

Reach

Downloads
677.1k
Stars
28

Details

License
Apache-2.0
First release
May 13, 2026
Releases
Version Released
4.0.0 major
3.1.3 patch
3.1.2 patch
3.1.1 patch
3.1.0 minor
3.0.0 major
2.0.0 major
1.9.2 patch
1.9.1 patch
1.9.0 minor
1.8.2 patch
1.8.1 patch
1.8.0 minor
1.7.0 minor
1.6.0 minor
1.5.4 patch
1.5.3 patch
1.5.2 patch
1.5.1 patch
1.5.0 minor
1.4.0 minor
1.3.2 patch
1.3.1 patch
1.2.1 patch
1.3.0-dev.1 pre
1.3.0 minor
1.2.0 initial
0.0.0-prerelease.0 pre