Promptfoo
MIT LLM evals & red-teaming CLI; Community free, 10k probes/mo
AI Agents / Memory & Evals · Freemium · 24.8k
What's Good
MIT. Compare prompts/models, score outputs, CI gates, red-team/OWASP-style scans. Runs locally — prompts stay on your machine. Works with OpenAI, Anthropic, Azure, Bedrock, Ollama, custom providers. Now part of OpenAI with stated MIT permanence.
The Catch
Vs DeepEval/Braintrust: YAML/CLI-first, not pytest-native Python or Braintrust SaaS. LOUD: Community caps red-team probes (~10k/mo); Enterprise/On-Prem are contact-sales (SSO, dashboards, dedicated runner). LLM API costs are yours. Acquisition governance is a multi-year bet.
Verdict
MIT eval/red-team CLI. Free core; probe cap + Enterprise sales wall.
Embed
[](https://stackgems.com/gems/promptfoo)