# Crawl4AI

> Apache-2.0 LLM-friendly crawler — markdown for agents

**Category:** AI Agents  
**Pricing:** open-source  
**URL:** https://crawl4ai.com  
**GitHub:** https://github.com/unclecode/crawl4ai  
**Added:** 2026-09-16

---

## The Hook

Crawl4AI turns the web into clean, LLM-ready markdown — a crawler shaped for RAG and agents, not a generic scrape SDK.

## What's Good

Apache-2.0 (unclecode/crawl4ai). Async Python crawler tuned for LLM pipelines: markdown extraction, chunking hooks, and agent-friendly output. Site at crawl4ai.com; pip-installable. Huge adoption (~84k★).

## The Catch

Vs Scrapling (adaptive scrape/crawl + MCP), Agent Reach (public-web CLI eyes), Browser Use/Stagehand (browser agents): Crawl4AI is an LLM-oriented crawler/extraction library, not a hosted scrape API, not a site-specific eyes CLI, and not a Playwright agent. You own proxies, blockers, rate limits, and site ToS.

## Verdict

Apache-2.0 LLM crawler. Markdown-for-agents vs Scrapling/Agent Reach/Browser Use/Stagehand.