Sign in Sign up
cargo

llm-multimodal

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

Activity

Latest release
2w ago
Total releases
13
Cadence
~13 days
Last 12 months
13

Reach

Downloads
127.8k
Stars
522

Details

License
Apache-2.0
First release
Jan 21, 2026
Releases
Version Released
1.11.0 minor
1.10.0 minor
1.9.0 minor
1.8.0 minor
1.7.1 patch
1.7.0 minor
1.6.0 minor
1.5.0 minor
1.4.0 minor
1.3.0 minor
1.2.0 minor
1.1.0 minor
1.0.0 initial