# TokenSpeed

> MIT LLM inference engine built for agent workloads

**Category:** AI Agents  
**Pricing:** open-source  
**URL:** https://github.com/lightseekorg/tokenspeed  
**GitHub:** https://github.com/lightseekorg/tokenspeed  
**Added:** 2026-10-16

---

## The Hook

TokenSpeed is an inference server tuned for agentic workloads — long multi-turn sessions, heavy prefix reuse and high per-user token speed — aiming for TensorRT-LLM-level performance with vLLM-style usability.

## What's Good

MIT (lightseekorg/tokenspeed), release v0.1.1. Kernels are a separate, pluggable subsystem with a public API, and the project publishes day-zero support and benchmarks for new open models on NVIDIA GB300. ★~2.5k.

## The Catch

Very early (v0.1.1), so expect breaking changes. The kernel work targets recent NVIDIA data-center GPUs such as Blackwell, so gains on older or consumer cards are unclear. It covers fewer models than vLLM — check yours is supported before you switch.

## Verdict

A fast-moving inference engine for agent traffic on top-end NVIDIA hardware — benchmark it beside vLLM before committing.