{"data":{"slug":"llama-swap","name":"llama-swap","tagline":"MIT one-binary proxy that hot-swaps local models on demand","category":"agents","categoryName":"AI Agents","url":"https://github.com/mostlygeek/llama-swap","github":"https://github.com/mostlygeek/llama-swap","pricing":"open-source","hook":"llama-swap puts one OpenAI- and Anthropic-compatible endpoint in front of llama.cpp, vLLM, stable-diffusion.cpp, whisper/audio.cpp and ComfyUI, starting and stopping the right server when a request names a model — so your agents can use many local models without all of them sitting in VRAM.","whatsGood":"MIT (mostlygeek/llama-swap, release v262). Go, zero dependencies: one binary + one YAML config. Covers chat/completions, responses, embeddings, audio, images, `v1/messages`, rerank; `ttl` auto-unload, profiles, concurrent-model swap matrix, API keys, `/metrics` for Prometheus, and a web UI with playground, token metrics, and log streaming. Docker unified images (CUDA 12/13, Vulkan), Homebrew, WinGet. ★~5.8k.","theCatch":"It is a process manager and proxy, not an inference engine — you still configure each upstream server, and a swap means a cold model load (seconds to minutes on big GGUFs). Config is the product: wrong `cmd` flags fail per model. Vs ollama (bundled runtime + model registry) and vllm (serving engine): llama-swap is the switchboard in front of whatever runtimes you already trust.","verdict":"MIT local-model switchboard when agents need many llama.cpp/vLLM models behind one endpoint on limited VRAM — skip if Ollama alone covers your models.","addedAt":"2026-10-05","featured":false,"domain":"AI Agents","subSpecialty":"Agents","subcategory":"Inference & Local Models","tags":["self-host","permissive","local-llm","voice","active"],"capabilities":[],"surfaces":["Self-host"],"ecosystem":["Multi-platform"],"licenseModel":"OSS/MIT","githubStars":6010,"starsPerWeek":56,"access":"free","archived":false,"repo":{"createdAt":"2024-10-04","pushedAt":"2026-10-15","license":"MIT","archived":false,"checkedAt":"2026-10-19"}},"related":[{"slug":"claude-code","name":"Claude Code","tagline":"Terminal-first coding agent with deep repository context"},{"slug":"cursor","name":"Cursor","tagline":"The AI-first code editor that understands your codebase"},{"slug":"devin","name":"Devin","tagline":"Autonomous AI engineer for defined engineering tasks"},{"slug":"stagehand","name":"Stagehand","tagline":"AI-native browser automation that mixes code and natural language"},{"slug":"browser-use","name":"Browser Use","tagline":"Open-source framework for LLM-driven browser agents"}],"links":{"self":"/api/gems/llama-swap","html":"/gems/llama-swap","markdown":"/gems/llama-swap.md","category":"/api/gems?category=agents","categoryPage":"/stacks/agents"}}