# llama-swap

> MIT one-binary proxy that hot-swaps local models on demand

**Category:** AI Agents  
**Pricing:** open-source  
**URL:** https://github.com/mostlygeek/llama-swap  
**GitHub:** https://github.com/mostlygeek/llama-swap  
**Added:** 2026-10-05

---

## The Hook

llama-swap puts one OpenAI- and Anthropic-compatible endpoint in front of llama.cpp, vLLM, stable-diffusion.cpp, whisper/audio.cpp and ComfyUI, starting and stopping the right server when a request names a model — so your agents can use many local models without all of them sitting in VRAM.

## What's Good

MIT (mostlygeek/llama-swap, release v262). Go, zero dependencies: one binary + one YAML config. Covers chat/completions, responses, embeddings, audio, images, `v1/messages`, rerank; `ttl` auto-unload, profiles, concurrent-model swap matrix, API keys, `/metrics` for Prometheus, and a web UI with playground, token metrics, and log streaming. Docker unified images (CUDA 12/13, Vulkan), Homebrew, WinGet. ★~5.8k.

## The Catch

It is a process manager and proxy, not an inference engine — you still configure each upstream server, and a swap means a cold model load (seconds to minutes on big GGUFs). Config is the product: wrong `cmd` flags fail per model. Vs ollama (bundled runtime + model registry) and vllm (serving engine): llama-swap is the switchboard in front of whatever runtimes you already trust.

## Verdict

MIT local-model switchboard when agents need many llama.cpp/vLLM models behind one endpoint on limited VRAM — skip if Ollama alone covers your models.