Tag
ci
Tools in this catalog tagged ci. Honest Catch, not a roundup.
7 tools · the Catch, not a ranking
- BraintrustEval-first LLM platform that blocks bad prompts in CI
Eval-first means observability is secondary—if your primary loop is debugging traces, look elsewhere. Learning curve to build eval datasets that actually represent production.
- GitHub ActionsCI/CD that lives where your code does
YAML syntax has a learning curve. Debugging failed workflows is frustrating. Costs scale with build minutes. Complex workflows become hard to maintain.
- DaggerApache-2.0 CI as containers, programmable in your language
You need Docker or a compatible engine. Rewriting mature CI YAML is a project, not an afternoon. Dagger Cloud is a separate product — this review does not invent its free tier.
- Woodpecker CIApache-2.0 container-native CI; self-host forever, no paid tier
Self-host only — not GitHub Actions managed UX. Smaller plugin ecosystem than Actions/Drone-era commercial. You own runners, secrets, and forge OAuth. Distinct from GitHub Actions and Buildkite hosted.
- PromptfooMIT LLM evals & red-teaming CLI; Community free, 10k probes/mo
Vs DeepEval/Braintrust: YAML/CLI-first, not pytest-native Python or Braintrust SaaS. LOUD: Community caps red-team probes (~10k/mo); Enterprise/On-Prem are contact-sales (SSO, dashboards, dedicated runner). LLM API costs are yours. Acquisition governance is a multi-year bet.
- GitleaksMIT secret scanner for git/dirs/stdin; CLI free, org Action needs free key
LOUD: maintainer marks feature-complete (security patches; focus shifting to Betterleaks). Official Action for GitHub org repos needs a free license key from gitleaks.io — personal accounts exempt. Not a full ASVS/SAST suite. False positives need allowlists.
- GrypeApache-2.0 SBOM-first vulnerability scanner from Anchore; free CLI
Vuln scanner — not Trivy's IaC/secrets swiss-army, not Cosign signing, not Anchore Enterprise governance UI. You still triage CVEs. Anchore monetizes Enterprise separately.