{"data":{"slug":"tokenspeed","name":"TokenSpeed","tagline":"MIT LLM inference engine built for agent workloads","category":"agents","categoryName":"AI Agents","url":"https://github.com/lightseekorg/tokenspeed","github":"https://github.com/lightseekorg/tokenspeed","pricing":"open-source","hook":"TokenSpeed is an inference server tuned for agentic workloads — long multi-turn sessions, heavy prefix reuse and high per-user token speed — aiming for TensorRT-LLM-level performance with vLLM-style usability.","whatsGood":"MIT (lightseekorg/tokenspeed), release v0.1.1. Kernels are a separate, pluggable subsystem with a public API, and the project publishes day-zero support and benchmarks for new open models on NVIDIA GB300. ★~2.5k.","theCatch":"Very early (v0.1.1), so expect breaking changes. The kernel work targets recent NVIDIA data-center GPUs such as Blackwell, so gains on older or consumer cards are unclear. It covers fewer models than vLLM — check yours is supported before you switch.","verdict":"A fast-moving inference engine for agent traffic on top-end NVIDIA hardware — benchmark it beside vLLM before committing.","addedAt":"2026-10-16","featured":false,"domain":"AI Agents","subSpecialty":"Inference","subcategory":"Inference & Local Models","tags":["self-host","permissive","local-llm","active","young-repo"],"capabilities":["inference","gpu"],"surfaces":["Self-host"],"ecosystem":["Python"],"licenseModel":"OSS/MIT","githubStars":2703,"starsPerWeek":114,"access":"free","archived":false,"repo":{"createdAt":"2026-05-06","pushedAt":"2026-10-19","license":"MIT","archived":false,"checkedAt":"2026-10-19"}},"related":[{"slug":"claude-code","name":"Claude Code","tagline":"Terminal-first coding agent with deep repository context"},{"slug":"cursor","name":"Cursor","tagline":"The AI-first code editor that understands your codebase"},{"slug":"devin","name":"Devin","tagline":"Autonomous AI engineer for defined engineering tasks"},{"slug":"stagehand","name":"Stagehand","tagline":"AI-native browser automation that mixes code and natural language"},{"slug":"browser-use","name":"Browser Use","tagline":"Open-source framework for LLM-driven browser agents"}],"links":{"self":"/api/gems/tokenspeed","html":"/gems/tokenspeed","markdown":"/gems/tokenspeed.md","category":"/api/gems?category=agents","categoryPage":"/stacks/agents"}}