TokenSpeed
MIT LLM inference engine built for agent workloads
AI Agents / Inference · OSS/MIT · 2.7k
Reviewed 2026-10-16 · last push 2026-10-19 · repo checked 2026-10-19
What's Good
MIT (lightseekorg/tokenspeed), release v0.1.1. Kernels are a separate, pluggable subsystem with a public API, and the project publishes day-zero support and benchmarks for new open models on NVIDIA GB300. ★~2.5k.
The Catch
Very early (v0.1.1), so expect breaking changes. The kernel work targets recent NVIDIA data-center GPUs such as Blackwell, so gains on older or consumer cards are unclear. It covers fewer models than vLLM — check yours is supported before you switch.
Verdict
A fast-moving inference engine for agent traffic on top-end NVIDIA hardware — benchmark it beside vLLM before committing.
Embed
Maintainer? Add the badge to your README. It is free, links to this review, and there is no paid version.
Markdown
[](https://stackgems.com/gems/tokenspeed)
HTML
<a href="https://stackgems.com/gems/tokenspeed"><img src="https://stackgems.com/badge/tokenspeed.svg" alt="TokenSpeed reviewed on Stack Gems" height="20"></a>
Related
AI Agents →- OllamaMIT local model runner — free forever; Pro cloud $20/mo
- vLLMApache-2.0 high-throughput LLM inference and serving engine
- KevApache-2.0 tiny Jev-like decision models you train and serve locally
- ReefApache-2.0 infra for continually self-improving agents
- MagnitudeApache-2.0 local inference desktop+CLI — consumer profiler for agents