# SMG (Shepherd Model Gateway)

> Apache-2.0 Rust gateway that routes OpenAI and Anthropic API traffic across vLLM, SGLang, TRT-LLM and cloud models

**Category:** AI Agents  
**Pricing:** open-source  
**URL:** https://lightseek.org/smg  
**GitHub:** https://github.com/smg-project/smg  
**Added:** 2026-10-21

---

## The Hook

SMG is a model-routing gateway written in Rust: one OpenAI- and Anthropic-compatible endpoint in front of self-hosted engines and cloud providers.

## What's Good

Apache-2.0 (smg-project/smg), release v1.13.0 (2026-10-15), commit on 2026-10-21. ★~660 (~14 a week). It manages worker lifecycles, routes with KV-cache awareness, and adds chat-history storage, MCP tooling, multi-tenant auth and observability. Ships via Docker and PyPI. Useful when you run several inference engines or GPUs and want one endpoint that balances across them.

## The Catch

It is the layer in front of an engine, not an engine. The SGLang gem serves the model; SMG routes traffic to SGLang, vLLM, TRT-LLM or cloud APIs. If you run one model on one box, you don't need it. The project is young (repo started November 2025) and small, so expect gaps against heavier gateways.

## Verdict

A serious open gateway for multi-engine LLM fleets; overkill for a single server.