Ragas

Apache-2.0 RAG/agent eval metrics — synthetic data + CI scorers

AI Agents / Agents · Apache-2.0 · 16.1k

Reviewed 2026-09-29 · last push 2026-02-24 · repo checked 2026-10-19

What's Good

Apache-2.0 (explodinggradients/ragas → vibrantlabsai). Python metrics for RAG (faithfulness, answer relevancy, context precision/recall) plus agent/trajectory scorers and synthetic test-data generation. Runs locally or in CI against your own LLM providers. Docs at ragas.io / docs.ragas.io. ★15,869 on GitHub 2026-09-29.

The Catch

Vs DeepEval (pytest-native agent/chat metrics + Confident AI cloud) and Promptfoo (YAML/CLI red-team): Ragas is RAG-first metric library + synthetic data, not a full red-team harness and not Confident AI dashboards. LLM-as-judge burns provider tokens; cloud extras (if any) are optional — core library stays OSS.

Verdict

Apache-2.0 RAG/agent eval metrics when faithfulness and context precision beat a hosted eval bill — not DeepEval pytest, not Promptfoo red-team.

Embed

Maintainer? Add the badge to your README. It is free, links to this review, and there is no paid version.

Markdown

[![Reviewed on Stack Gems](https://stackgems.com/badge/ragas.svg)](https://stackgems.com/gems/ragas)

HTML

<a href="https://stackgems.com/gems/ragas"><img src="https://stackgems.com/badge/ragas.svg" alt="Ragas reviewed on Stack Gems" height="20"></a>

AI Agents →

Related