Testing tools

Free / OSS · OSS/MIT · 50.2k GitHub stars

VerdictOne fast tool to replace your entire lint toolchain.

CatchNot every rule from every tool is implemented. Some teams have strong opinions about specific linter behaviors.

Free / OSS · OSS/MIT · 96.3k GitHub stars

VerdictMIT JS toolkit. Free forever; Node parity is the real catch.

CatchNode-compat is not 100%. Joined Anthropic — watch ecosystem/governance noise, but license remains MIT. Not Deno. Some packages still assume Node quirks.

Free / OSS · OSS/MIT · 17.3k GitHub stars

VerdictMIT Vite test runner. Free forever; Vite itself is the bundler gem.

CatchVs Vite: Vite-native test runner; not the bundler. Not full E2E browser suites. Closest when Vite is already the toolchain.

Free / OSS · Apache-2.0 · 98k GitHub stars

VerdictApache-2.0 browser library. Free; MCP wrapper is a separate gem.

CatchVs Playwright MCP gem: library/CLI lane; not the MCP wrapper. Heavier than unit tests. Browser downloads and CI images are real ops. Not Stagehand/Browserbase.

Free / OSS · OSS/MIT · 10.7k GitHub stars

VerdictMIT MailHog successor for local/CI. Free forever; not a mailer.

CatchSink only — not delivery. Not Resend SaaS, not Postal/Listmonk outbound. Do not point production MX here. Distinct from Resend/Postal/Listmonk.

Free / OSS · OSS/MIT · 32k GitHub stars

VerdictMIT shell history. Free forever CLI + self-host sync; hosted sync stays optional.

CatchShell history tool — not a prompt (Starship), not a jumper (zoxide), not Dotfiles sync. Desktop/Hub team features are a separate product line (Hub not self-hostable). Atuin AI is free while in testing — treat that as temporary.

Free / OSS · Apache-2.0 · 19.3k GitHub stars

VerdictApache-2.0 HTTP test files. Free forever; commit the session, run it in CI.

CatchVs xh/HTTPie (interactive HTTP CLIs) and Bruno/Hoppscotch (API *clients*): Hurl is the *plain-text test runner*, not a REPL client and not a collections GUI. You author `.hurl` files; it is not Postman-shaped collaboration. libcurl semantics apply.

Free / OSS · AGPL-3.0 · 32k GitHub stars

VerdictAGPL-3.0 scriptable load. Free unmodified CLI/CI; Cloud k6 is the paid seat.

CatchAGPL-3.0 network/copyleft — modifications you serve or distribute can trigger source obligations; do not treat k6 as MIT. Grafana Cloud k6 is a separate paid hosted product (execution, analysis, collaboration). Vs oha (MIT TUI hammer): k6 is the *scripted/scenario* suite, not a live ratatui chart. Point it at staging you own.

Free / OSS · Apache-2.0 · 15.6k GitHub stars

VerdictApache-2.0 local kube. Free forever; spin a cluster in Docker, tear it down in CI.

CatchVs k3d/k3s (lighter single-binary k8s), colima (container runtime host), and minikube: kind is *Kubernetes-in-containers*, not a production control plane and not a substitute for cloud quotas. Needs a working container runtime; multi-node “realism” is still fake under one Docker host.

Free / OSS · OSS/MIT · 108k GitHub stars

VerdictMIT production skill pack. Reach for it when you want explicit /spec→/ship gates across many harnesses.

CatchPer-skill installs can miss shared `references/` checklists — prefer whole-repo or full pack install. Still markdown discipline, not a runtime that forces compliance if the host ignores skills. Vs Superpowers (session-hijacking SDLC methodology) and knowledge-work-plugins (Anthropic official packs): this is the Google-flavored lifecycle library you pick skill-by-skill, not an automatic brainstorm→subagent loop.

Free / OSS · Apache-2.0 · 2.3k GitHub stars

VerdictApache-2.0 MCP test platform. Use for real-client MCP debugging/evals; watch the proprietary evals carve-out.

CatchVs MCP Inspector (official MIT debugger): MCPJam is the broader test/eval platform with hosted gravity, not only the reference inspector. Some `/server/services` and evals paths are separately licensed proprietary portions — treat the free OSS inspector path as the bar, not every hosted feature.

Free / OSS · Apache-2.0 · 11.3k GitHub stars

VerdictApache agentic e2e when flaky selectors cost more than tokens — record-and-replay keeps CI cheap; pin minors and turn telemetry off.

CatchTelemetry: the CLI sends anonymous usage data by default — `npx e2e telemetry disable` or `E2E_TELEMETRY_DISABLED=1`. Pre-1.0: APIs and config can change between minors. Built by TesterArmy, a commercial agentic-testing platform; Kernel hosted browsers and EAS simulators are optional paid add-on engines. Vs stagehand (browser automation SDK) and playwright (deterministic runner): e2e is a test framework that mixes agent steps with assertions and caches them.

Free / OSS · AGPL-3.0 · 28.6k GitHub stars

VerdictThe scanner to run when you need to know which leaked keys are live — mind the AGPL and the outbound verification calls.

CatchLicense first: AGPL-3.0 is copyleft, not permissive — check it with your legal team before embedding or modifying it in a product you distribute or offer as a service. Verifying a secret calls the real provider’s API with it, so scans make outbound requests and show up in that provider’s logs. Continuous monitoring of Slack, Jira, Confluence and similar tools is in the paid Enterprise product; the open-source CLI covers one-off and CI scans. Vs gitleaks (MIT, fast regex scanning, no live verification): TruffleHog tells you which leaks still work, at the cost of a copyleft license and network calls.

Free / OSS · OSS/MIT · 222 GitHub stars

VerdictSelf-hosted GitHub Actions with real VM isolation — promising, but young; try it beside your existing CI first.

CatchYou run a server, not a one-shot CLI — expose it on a tailnet, tunnel or your own domain and keep it patched. It is a small, young project (first commit mid-2026), so expect rough edges and pin versions. Vs act (runs workflows locally in Docker containers with one command): preloop is the heavier choice when you want real isolation and a standing CI service you own.

Free / OSS · OSS/MIT · 3.3k GitHub stars

VerdictDrop-in fast pixel diff for screenshot tests and agent UI checks — bring your own baseline workflow.

CatchIt only answers whether two images differ and where: there is no baseline storage, review UI or approval flow — you still need your test runner or a service for that. Diff output is PNG only. Pixel-exact comparison means fonts, GPU rendering and animations can make tests flaky unless you pin the browser environment and set thresholds or ignore regions.

Free / OSS · Apache-2.0 · 10.7k GitHub stars

VerdictThe default toolkit for new Solidity projects — budget time to port tests if you're moving off Hardhat.

CatchEVM chains only: it does nothing for Solana or other non-EVM chains. Tests are written in Solidity rather than JavaScript or TypeScript, which is a real shift for teams coming from Hardhat and their existing test suites. It is a toolkit, not an audit: passing fuzz tests are not a security review.

Free / OSS · OSS/MIT · 13.1k GitHub stars

VerdictThe most batteries-included Python browser framework — powerful, but use the stealth parts responsibly.

CatchA large share of the README is about bypassing bot detection and CAPTCHAs, which can break site terms — use it on sites you own or are allowed to automate. The API surface is huge, so pick one style and stick to it. For new JavaScript projects, Playwright is the more common default.

Free / OSS · Apache-2.0 · 5k GitHub stars

VerdictWorth a look if you want agent tests and an LLM gateway in the same tool as your traces.

CatchIt tries to be four products at once, so expect a broad surface that overlaps with Langfuse, Opik and LiteLLM. The production self-host is more than a single Node process. The managed LangWatch Cloud is the commercial path, and README links lean that way.

Free / OSS · MPL-2.0 · 9.1k GitHub stars

VerdictA solid OSS load tester, especially for Playwright-based browser tests.

CatchThe hosted Artillery Cloud dashboard is a separate paid product; the CLI is the free part. Running at scale bills your own AWS account. k6 is the main alternative if you prefer scripting in JavaScript only.

Free / OSS · OSS/MIT · 29.4k GitHub stars

VerdictThe most complete monorepo toolkit for JavaScript teams.

CatchRemote caching and distributed CI run through Nx Cloud, a paid service with a free tier. Plugins and generators add their own conventions and upgrade churn. For a small repo, Turborepo or plain workspaces are lighter.

Free / OSS · Apache-2.0 · 2.5k GitHub stars

VerdictA serious, offline repo map for agents — trust the evals doc over the marketing.

CatchThe README is very long and leans on a field report written by a model about one engagement, which is not a controlled measurement; the repo has a separate EVALS doc, so read that instead. The one-line install pipes a script from GitHub into bash. Young and pre-1.0, and languages without a grammar fall back to weaker results, which it labels. It overlaps with Serena and Repomix for repo context.

Free / OSS · OSS/MIT · 15.1k GitHub stars

VerdictThe most complete free bridge between coding agents and the Unity Editor — commit before you prompt.

CatchNeeds Python 3.10+ via uv alongside Unity. An agent with editor access can make a mess of a scene or project, so keep the project in version control and review changes. Maintained and sponsored by Aura, whose paid Unity/Unreal assistant is pitched in the README; the MCP server itself is free. Not affiliated with Unity Technologies.

Free / OSS · Apache-2.0 · 16.1k GitHub stars · quiet 180d+

VerdictApache-2.0 RAG/agent eval metrics when faithfulness and context precision beat a hosted eval bill — not DeepEval pytest, not Promptfoo red-team.

CatchVs DeepEval (pytest-native agent/chat metrics + Confident AI cloud) and Promptfoo (YAML/CLI red-team): Ragas is RAG-first metric library + synthetic data, not a full red-team harness and not Confident AI dashboards. LLM-as-judge burns provider tokens; cloud extras (if any) are optional — core library stays OSS.