Promptfoo

MIT LLM evals & red-teaming CLI; Community free, 10k probes/mo

AI Agents / Memory & Evals · Freemium · 26.2k

Reviewed 2026-09-04 · last push 2026-10-21 · repo checked 2026-10-21

What's Good

MIT. Compare prompts/models, score outputs, CI gates, red-team/OWASP-style scans. Runs locally — prompts stay on your machine. Works with OpenAI, Anthropic, Azure, Bedrock, Ollama, custom providers. Now part of OpenAI with stated MIT permanence.

The Catch

Vs DeepEval/Braintrust: YAML/CLI-first, not pytest-native Python or Braintrust SaaS. Community caps red-team probes (~10k/mo); Enterprise/On-Prem are contact-sales (SSO, dashboards, dedicated runner). LLM API costs are yours. Acquisition governance is a multi-year bet.

Verdict

MIT eval/red-team CLI. Free core; probe cap + Enterprise sales wall.

Embed

Maintainer? Add the badge to your README. It is free, links to this review, and there is no paid version.

Markdown

[![Reviewed on Stack Gems](https://stackgems.com/badge/promptfoo.svg)](https://stackgems.com/gems/promptfoo)

HTML

<a href="https://stackgems.com/gems/promptfoo"><img src="https://stackgems.com/badge/promptfoo.svg" alt="Promptfoo reviewed on Stack Gems" height="20"></a>

AI Agents →

Related