Crawlee
Apache-2.0 Node.js crawling and scraping library from Apify, HTTP or headless browser
AI Agents / Web Scraping · Apache-2.0 · 26.2k
Reviewed 2026-10-10 · last push 2026-10-19 · repo checked 2026-10-19
What's Good
Apache-2.0 (apify/crawlee), release v3.18.2. `npx crawlee create my-crawler` scaffolds a TypeScript project. Swap between a fast HTTP crawler and a full browser crawler with the same API, with browser fingerprinting and autoscaling built in. Runs anywhere; deploying to Apify is optional. ★~26k.
The Catch
Needs Node 22.13 or newer. The Python version is a separate project with its own feature set, so do not assume parity. Heavy production crawling usually needs paid proxies to avoid blocks — the library rotates them, it does not supply them. Vs apify-mcp (gives an agent access to ready-made Apify scrapers) and playwright (the browser automation underneath): Crawlee is the crawler framework you build your own scraper on.
Verdict
The default Node scraping framework when a one-off Playwright script has turned into a crawler — budget for proxies.
Embed
Maintainer? Add the badge to your README. It is free, links to this review, and there is no paid version.
Markdown
[](https://stackgems.com/gems/crawlee)
HTML
<a href="https://stackgems.com/gems/crawlee"><img src="https://stackgems.com/badge/crawlee.svg" alt="Crawlee reviewed on Stack Gems" height="20"></a>