Sign in Sign up
cargo

llm-tokenizer

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

cargo View on Cargo github View on Github

Activity

Latest release
1w ago
Total releases
16
Cadence
~14 days
Last 12 months
16

Reach

Downloads
2.1M
Stars
690

Details

License
Apache-2.0
First release
Jan 06, 2026
Releases
Version Released
1.10.0 minor
1.9.0 minor
1.8.0 minor
1.7.0 minor
1.6.0 minor
1.5.0 minor
1.4.2 patch
1.4.1 patch
1.4.0 minor
1.3.2 patch
1.3.1 patch
1.3.0 minor
1.2.0 minor
1.1.0 minor
1.0.0 major
0.0.1 initial