# SGLang

> Apache-2.0 high-throughput LLM and multimodal serving

**Category:** AI Agents  
**Pricing:** open-source  
**URL:** https://www.sglang.io  
**GitHub:** https://github.com/sgl-project/sglang  
**Added:** 2026-10-13

---

## The Hook

SGLang is an open-source serving framework for language, vision-language and diffusion models, tuned for agentic workloads, large-scale serving and RL rollouts.

## What's Good

Apache-2.0 (sgl-project/sglang), release v0.5.21, hosted by the non-profit LMSYS. Free path: `docker pull lmsysorg/sglang:latest` or `uv pip install sglang`, then pick a launch command from the Cookbook for your model and hardware. Supports NVIDIA, AMD Instinct, Google TPU, Intel GPU/CPU, Apple Silicon via Metal/MLX and Huawei Ascend. Image and video generation (SGLang Diffusion) ships in the same package. Orchestrators such as llm-d, Ray Serve and NVIDIA Dynamo can run it as the engine. ★~37k.

## The Catch

A GPU serving engine, not a laptop chat app — expect driver, CUDA/ROCm and multi-GPU ops, and you pay for the hardware. Install currently needs `--prerelease=allow`. Vs vLLM (same lane): both are open, OpenAI-compatible serving engines — benchmark your own model and traffic rather than trusting either side's charts. Vs llama.cpp: SGLang is for many concurrent users on accelerators; llama.cpp is for one machine.

## Verdict

A top-tier open LLM serving engine beside vLLM — free software, the bill is GPUs; benchmark both on your workload.