{"data":{"slug":"nvidia-dynamo","name":"NVIDIA Dynamo","tagline":"Apache-2.0 datacenter-scale LLM inference orchestration above vLLM, SGLang and TensorRT-LLM","category":"agents","categoryName":"AI Agents","url":"https://docs.nvidia.com/dynamo/","github":"https://github.com/ai-dynamo/dynamo","pricing":"open-source","hook":"Dynamo is the orchestration layer above inference engines: it keeps vLLM, SGLang or TensorRT-LLM as the engine and turns a cluster of GPUs into one coordinated, OpenAI-compatible serving system with disaggregated prefill/decode, KV-aware routing, KV cache offload and SLA-driven autoscaling.","whatsGood":"Apache-2.0 (ai-dynamo/dynamo; a few test-data files are MIT). Free path: `pip install ai-dynamo` or pull the prebuilt runtime containers from NVIDIA NGC, then start a frontend plus a vLLM or SGLang worker. KV-aware routing avoids recomputing shared prefixes, the KV Block Manager spills cache to CPU, SSD or object storage, and the Planner right-sizes prefill and decode pools against latency targets. Kubernetes path includes a Gateway API Inference Extension plugin. The 1.0 line adds per-request hints for agentic workloads, and the repo ships agent skills for deploying and debugging Dynamo. Rust core, Python extension points. ★~8.2k.","theCatch":"Built for multi-GPU, multi-node serving — the README itself says one model on one GPU only needs the engine (see vLLM, or llama-swap for a local model switcher). Realistically NVIDIA hardware and Kubernetes; the headline speedups are vendor and partner benchmarks on GB200/GB300-class racks, so measure on your own GPUs. Lots of moving parts (frontend, router, workers, planner, discovery backend) and a fast release cadence: pin a version. Vs vLLM Semantic Router (model choice per request) and plain vLLM: Dynamo is the cluster scheduler, not a model router or an engine.","verdict":"Apache-2.0 cluster-level LLM serving when you run vLLM/SGLang/TRT-LLM across many GPUs — free, but overkill below multi-GPU scale and benchmarks are vendor-run.","addedAt":"2026-10-06","featured":false,"domain":"AI Agents","subSpecialty":"Inference","subcategory":"Inference & Local Models","tags":["self-host","permissive","license-unclear","local-llm","workflows","active"],"capabilities":["inference","llm-serving","kubernetes"],"surfaces":["Self-host"],"ecosystem":["Rust","Python"],"licenseModel":"Apache-2.0","githubStars":8329,"starsPerWeek":98,"access":"free","archived":false,"repo":{"createdAt":"2025-03-03","pushedAt":"2026-10-19","license":"NOASSERTION","archived":false,"checkedAt":"2026-10-19"}},"related":[{"slug":"claude-code","name":"Claude Code","tagline":"Terminal-first coding agent with deep repository context"},{"slug":"cursor","name":"Cursor","tagline":"The AI-first code editor that understands your codebase"},{"slug":"devin","name":"Devin","tagline":"Autonomous AI engineer for defined engineering tasks"},{"slug":"stagehand","name":"Stagehand","tagline":"AI-native browser automation that mixes code and natural language"},{"slug":"browser-use","name":"Browser Use","tagline":"Open-source framework for LLM-driven browser agents"}],"links":{"self":"/api/gems/nvidia-dynamo","html":"/gems/nvidia-dynamo","markdown":"/gems/nvidia-dynamo.md","category":"/api/gems?category=agents","categoryPage":"/stacks/agents"}}