smg-grpc-servicer
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across SGLang, vLLM, TRT-LLM, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
- Latest release
- 6h ago
- Releases
- 6
- Known CVEs
- 0
- First release
- Mar 05, 2026
- License
- Apache-2.0
Repository
Source
- Stars
- 322
- Forks
- 94
- Open issues
- 65
- Language
- Rust
- anthropic
- claude
- gemini
- inference-gateway
- llm
- openai
- routing
- sglang
Security score
No OpenSSF Scorecard available for this repository.
Packages from this repo
Insights
Activity
- Total releases
- 6
- Last 12 months
- 6
- Cadence
- ~16 days
- Dependencies
- 8
Releases per month
last 12 monthsRelease mix
- minor 1
- patch 4
6
releases
Dependencies
Depends on
0.5.4-
grpcio
>=1.78.0
-
grpcio-health-checking
>=1.78.0
-
grpcio-reflection
>=1.78.0
-
mlx
>=0.22.0
-
mlx-lm
>=0.22.0
-
sglang
>=0.5.10
-
smg-grpc-proto
>=0.4.9
-
vllm
>=0.19.0
Used by
Nothing tracked depends on this yet.
Releases
| Version | Released | |
|---|---|---|
0.5.4
patch
| ||
0.5.3
patch
| ||
0.5.2
patch
| ||
0.5.1
patch
| ||
0.5.0
minor
| ||
0.4.2
initial
|