{"data":{"slug":"llama-cpp","name":"llama.cpp","tagline":"MIT local LLM inference in C/C++ on almost any hardware","category":"agents","categoryName":"AI Agents","url":"https://llama.app","github":"https://github.com/ggml-org/llama.cpp","pricing":"open-source","hook":"llama.cpp is the C/C++ inference engine underneath much of the local-model world: install it, pull a GGUF model from Hugging Face, and `llama serve` gives you an OpenAI-compatible API plus a built-in web UI.","whatsGood":"MIT (ggml-org/llama.cpp), release v0.6.0, plus nightly builds. One-line installers (curl or PowerShell), prebuilt binaries, Docker, or build from source. Plain C/C++ with no dependencies; Apple Silicon is first-class via Metal, plus CUDA, HIP (AMD), Vulkan, SYCL, WebGPU and more. 1.5- to 8-bit quantization, and CPU+GPU hybrid inference for models bigger than your VRAM. Handles vision models too, and GBNF grammars constrain output. ★~131k.","theCatch":"It runs GGUF models, so you convert or download quantized weights, and model licences are separate — check the model card before commercial use. Lots of knobs: speed depends on backend, quant and offload settings, so expect to tune. Vs Ollama (friendlier model manager on similar ground): pick llama.cpp when you want the engine and its flags directly. Vs vLLM or SGLang: those are GPU serving engines for many concurrent users; llama.cpp is the one for laptops, edge boxes and single-user servers.","verdict":"The default local LLM engine — free, runs nearly anywhere, but you own the model choice and the tuning.","addedAt":"2026-10-13","featured":false,"domain":"AI Agents","subSpecialty":"Local Inference","subcategory":"Inference & Local Models","tags":["api","web-ui","self-host","permissive","local-llm","active","fast-stars"],"capabilities":["inference","local","llm-serving"],"surfaces":["API","Self-host"],"ecosystem":["Multi-platform"],"licenseModel":"OSS/MIT","githubStars":131654,"starsPerWeek":698,"access":"free","archived":false,"repo":{"createdAt":"2023-03-10","pushedAt":"2026-10-19","license":"MIT","archived":false,"checkedAt":"2026-10-19"}},"related":[{"slug":"claude-code","name":"Claude Code","tagline":"Terminal-first coding agent with deep repository context"},{"slug":"cursor","name":"Cursor","tagline":"The AI-first code editor that understands your codebase"},{"slug":"devin","name":"Devin","tagline":"Autonomous AI engineer for defined engineering tasks"},{"slug":"stagehand","name":"Stagehand","tagline":"AI-native browser automation that mixes code and natural language"},{"slug":"browser-use","name":"Browser Use","tagline":"Open-source framework for LLM-driven browser agents"}],"links":{"self":"/api/gems/llama-cpp","html":"/gems/llama-cpp","markdown":"/gems/llama-cpp.md","category":"/api/gems?category=agents","categoryPage":"/stacks/agents"}}