# vLLM Semantic Router

> Apache-2.0 Mixture-of-Models router — pick or compose the model path per request

**Category:** AI Agents  
**Pricing:** open-source  
**URL:** https://vllm-sr.ai  
**GitHub:** https://github.com/vllm-project/semantic-router  
**Added:** 2026-10-05

---

## The Hook

vLLM Semantic Router sits between your OpenAI/Anthropic-compatible endpoint and your model fleet and decides, per request, which model path to run — direct, specialist, local, cascade, or collaborative — from signals like intent, difficulty, modality, identity, and risk.

## What's Good

Apache-2.0 (vllm-project/semantic-router, v0.4.0; PyPI `vllm-sr` 0.4.0). Go core in the vLLM org. Stable entrypoints resolve to “recipes” that can attach retrieval, memory, tool filtering, caching, safety checks, and verification. Works with Envoy and existing inference runtimes rather than replacing them; Docker, Kubernetes, and hardware paths documented; public playground. ★~6k.

## The Catch

Platform, not a one-file proxy: it coordinates alongside Envoy and your model servers, so budget real ops time; the docs are written for heterogeneous fleets (cloud/data center/edge) more than a two-model hobby setup. Install is a `curl | bash` script (pip/uv paths exist — prefer those if you audit installs). Vs litellm / bifrost (multi-provider gateways) and weave-router (ELv2 embedder router): this is Apache policy-driven semantic routing that composes model paths.

## Verdict

Apache MoM router when one endpoint must route across many models and hardware by intent and cost — overkill if LiteLLM fallbacks already cover you.