{"data":{"slug":"sglang","name":"SGLang","tagline":"Apache-2.0 high-throughput LLM and multimodal serving","category":"agents","categoryName":"AI Agents","url":"https://www.sglang.io","github":"https://github.com/sgl-project/sglang","pricing":"open-source","hook":"SGLang is an open-source serving framework for language, vision-language and diffusion models, tuned for agentic workloads, large-scale serving and RL rollouts.","whatsGood":"Apache-2.0 (sgl-project/sglang), release v0.5.21, hosted by the non-profit LMSYS. Free path: `docker pull lmsysorg/sglang:latest` or `uv pip install sglang`, then pick a launch command from the Cookbook for your model and hardware. Supports NVIDIA, AMD Instinct, Google TPU, Intel GPU/CPU, Apple Silicon via Metal/MLX and Huawei Ascend. Image and video generation (SGLang Diffusion) ships in the same package. Orchestrators such as llm-d, Ray Serve and NVIDIA Dynamo can run it as the engine. ★~37k.","theCatch":"A GPU serving engine, not a laptop chat app — expect driver, CUDA/ROCm and multi-GPU ops, and you pay for the hardware. Install currently needs `--prerelease=allow`. Vs vLLM (same lane): both are open, OpenAI-compatible serving engines — benchmark your own model and traffic rather than trusting either side's charts. Vs llama.cpp: SGLang is for many concurrent users on accelerators; llama.cpp is for one machine.","verdict":"A top-tier open LLM serving engine beside vLLM — free software, the bill is GPUs; benchmark both on your workload.","addedAt":"2026-10-13","featured":false,"domain":"AI Agents","subSpecialty":"Inference","subcategory":"Inference & Local Models","tags":["self-host","permissive","active"],"capabilities":["inference","llm-serving","gpu"],"surfaces":["Self-host"],"ecosystem":["Python"],"licenseModel":"Apache-2.0","githubStars":37296,"starsPerWeek":257,"access":"free","archived":false,"repo":{"createdAt":"2024-01-08","pushedAt":"2026-10-19","license":"Apache-2.0","archived":false,"checkedAt":"2026-10-19"}},"related":[{"slug":"claude-code","name":"Claude Code","tagline":"Terminal-first coding agent with deep repository context"},{"slug":"cursor","name":"Cursor","tagline":"The AI-first code editor that understands your codebase"},{"slug":"devin","name":"Devin","tagline":"Autonomous AI engineer for defined engineering tasks"},{"slug":"stagehand","name":"Stagehand","tagline":"AI-native browser automation that mixes code and natural language"},{"slug":"browser-use","name":"Browser Use","tagline":"Open-source framework for LLM-driven browser agents"}],"links":{"self":"/api/gems/sglang","html":"/gems/sglang","markdown":"/gems/sglang.md","category":"/api/gems?category=agents","categoryPage":"/stacks/agents"}}