What are the best free alternatives to Promptfoo?

Langfuse, DeepEval, Engram, AX Check, iFixAi. All are reviewed on Stack Gems in the same category (agent memory tools); 8 free or open source.

Is there an open-source alternative to Promptfoo?

Yes: Langfuse (Open-Core), DeepEval (Apache-2.0), Engram (OSS/MIT), iFixAi (Apache-2.0).

Alternatives · Memory & Evals

Promptfoo alternatives (2026)

Promptfoo: MIT eval/red-team CLI. Free core; probe cap + Enterprise sales wall.

Its Catch: Vs DeepEval/Braintrust: YAML/CLI-first, not pytest-native Python or Braintrust SaaS.

Below: 8 free or free-tier tools from the same category (agent memory tools), free and open source first, then the closest match by shared tags. Each keeps its own Catch.

1.LangfuseOpen-source LLM observability you can self-host
Free / OSS · Open-Core · 36k GitHub stars
VerdictThe LLM observability platform for teams who want to own their data.
CatchLicense: the core is MIT, but anything under the ee/ directories is under a separate enterprise license — check which features you rely on before calling your deployment fully MIT. Self-hosting means self-operating—you are running Postgres, ClickHouse, Redis, and the app. Observability-first means evals are layered on, not central.
2.DeepEvalApache-2.0 pytest-style LLM/agent evals; Confident AI is the paid cloud
Free / OSS · Apache-2.0 · 18.9k GitHub stars
VerdictApache-2.0 LLM unit tests. Free locally; Confident AI starts free then $200.
CatchOSS is the runner — collaboration/dashboards/monitoring are Confident AI (Free: 2 seats/1 project/5 runs/wk; Starter $200/mo; Team $2k/mo). LLM-as-judge burns provider tokens. Vs Ragas: broader agent/chat coverage, not RAG-only.
3.EngramMIT persistent agent memory — MCP, CLI, TUI, SQLite
Free / OSS · OSS/MIT · 7.4k GitHub stars
VerdictMIT agent memory primitive. MCP/CLI/TUI on SQLite; not a full agent framework.
CatchVs Mem0 / Letta / full agent frameworks: Engram is the memory primitive (store + retrieve + surfaces), not an orchestration framework or coding agent. “local or cloud” branding means optional hosted/sync gravity — day-one stays OSS you run. You still decide what gets written and when.
4.AX CheckHosted free agent-experience checklist + recorded coding-agent sessions
Free / OSS · OSS/MIT
VerdictFree hosted AX smoke test. Grades how agents use your product, not how good your model is.
CatchGrades are provisional and technical-only — treat the report as a useful first pass, not a product-wide quality certificate. Vs Promptfoo / DeepEval: AX Check grades product AX (docs, CLI, MCP, SDK), not model quality, eval scores, or safety behavior. Hosted free is convenient; verify the session evidence and rerun after interface changes.
5.iFixAiApache-2.0 agents eval/audit — pip install, premium ≠ paywall
Free / OSS · Apache-2.0 · 29.7k GitHub stars
VerdictApache agents eval/audit when you want pip-installable scoring without an eval SaaS gate — not Braintrust/Langfuse/Ragas/Promptfoo’s lanes; premium ≠ required.
CatchVs Braintrust / Langfuse / Ragas / Promptfoo: Braintrust is eval-first commercial; Langfuse is MIT obs/self-host; Ragas is Apache RAG/agent metrics + CI scorers; Promptfoo is OSS prompt/agent eval — iFixAi’s catch is agents eval/audit as an installable audit kit, not Braintrust seats, not Langfuse traces, not Ragas metrics alone, not Promptfoo prompt suites. Stars grow fast (~47 a week): judge the audit loop, not the counter.
6.Obsidian MindMIT Obsidian vault as persistent agent memory
Free / OSS · OSS/MIT · 5.4k GitHub stars
VerdictThe most complete plain-Markdown memory setup for Claude Code and friends — free, but opinionated and Obsidian-bound.
CatchIt is opinionated about work life — standups, 1:1s, brag docs, incident capture — so it fits people who want an agent-run work journal, not just a project memory file. Needs Obsidian 1.12+ (a free but closed-source app) with its CLI enabled, plus Node 22+. QMD downloads about 2 GB of local models on first use. Hooks run on every session and message, so read them before you trust a vault. Vs mem0 or Letta: those are memory layers you build into an app; this is a personal vault for your own coding agent.
7.BeadsMIT issue-tracker memory for coding agents, on Dolt
Free / OSS · OSS/MIT · 28k GitHub stars
VerdictThe task memory to give a coding agent once its plans outgrow a markdown file — try --stealth first on repos you share.
CatchIt runs on Dolt, a version-controlled SQL database: the default embedded mode allows a single writer, and several concurrent writers need a separate dolt sql-server. bd init edits AGENTS.md and installs Claude Code, Codex and Cursor integrations by default — pass --stealth or --skip-agents on shared repos. Upgrades that cross a schema migration need one clone to migrate and the rest to bootstrap, and the HTTP backend is still experimental. Stars have grown fast (about 530 a week since the repo opened in Oct 2025), so judge it on a trial run, not the count.
8.Mem0Memory infrastructure for AI agents that actually persists
Free / OSS · Freemium · 67.4k GitHub stars
VerdictThe memory layer most agent teams should use instead of building.
CatchExternal dependency for a core capability. Async processing adds latency. Platform optimizations are not in the open-source SDK. You are trusting Mem0 with potentially sensitive user data.

FAQ

What are the best free alternatives to Promptfoo?: Langfuse, DeepEval, Engram, AX Check, iFixAi. All are reviewed on Stack Gems in the same category (agent memory tools); 8 free or open source.
What is the catch with Promptfoo?: Vs DeepEval/Braintrust: YAML/CLI-first, not pytest-native Python or Braintrust SaaS.
Is there an open-source alternative to Promptfoo?: Yes: Langfuse (Open-Core), DeepEval (Apache-2.0), Engram (OSS/MIT), iFixAi (Apache-2.0).

Promptfoo alternatives (2026)

FAQ

More