aphrody-llama-server
Managed local inference: a supervisor for llama.cpp's `llama-server`, driven as an external binary (never linked). Reads GGUF facts, probes what the installed build supports, plans the launch (context, KV precision, MoE/layer placement, LoRA), runs the server under a detached idle-timeout daemon that holds the GPU lease, and gives clients attach-or-spawn access to an OpenAI-compatible endpoint plus the tokenize / LoRA hot-swap / props side channels.
Activity
- Latest release
- 6d ago
- Total releases
- 1
- Cadence
- —
- Last 12 months
- 1
Reach
- Downloads
- 0
Details
- License
- Apache-2.0
- First release
- Oct 23, 2026
Releases