uzu

MIT on-device inference engine for Apple Silicon apps

AI Agents / Local Inference · OSS/MIT · 2.4k

Reviewed 2026-10-13 · last push 2026-10-18 · repo checked 2026-10-19

What's Good

MIT (trymirai/uzu), release 0.6.3. Native Rust crate; Swift bindings via uniffi, Python via pyo3, TypeScript via napi-rs. Metal and CPU backends, using unified memory on Apple devices; targets macOS, iOS and Mac Catalyst. One high-level API covers local chat sessions, structured output, tool calls, and optional cloud models through your own OpenAI key. A CLI can browse, download, benchmark and serve models over /v1/chat/completions. ★~2.2k.

The Catch

Apple-first: Linux, Windows and WASM targets are still listed as in progress. It uses its own model format — you download Mirai's converted models or export one yourself with their lalamo tool, so you cannot drop in an arbitrary GGUF. The Rust quick start pulls from the git main branch. Backed by a company (Mirai). Vs Rapid-MLX (a local server for coding agents on a Mac): uzu is an embeddable engine for shipping AI inside iOS and macOS apps.

Verdict

The pick for on-device models inside an iOS or macOS app — MIT and free, but Apple-only for now and tied to its own model format.

Embed

Maintainer? Add the badge to your README. It is free, links to this review, and there is no paid version.

Markdown

[![Reviewed on Stack Gems](https://stackgems.com/badge/uzu.svg)](https://stackgems.com/gems/uzu)

HTML

<a href="https://stackgems.com/gems/uzu"><img src="https://stackgems.com/badge/uzu.svg" alt="uzu reviewed on Stack Gems" height="20"></a>

AI Agents →

Related