{"data":{"slug":"crawlee","name":"Crawlee","tagline":"Apache-2.0 Node.js crawling and scraping library from Apify, HTTP or headless browser","category":"libraries","categoryName":"Libraries","url":"https://crawlee.dev","github":"https://github.com/apify/crawlee","pricing":"open-source","hook":"Crawlee wraps the boring parts of scraping — request queues, retries, proxy rotation, session handling, storage — around Cheerio, Playwright or Puppeteer crawlers, so you write the page logic and not the plumbing.","whatsGood":"Apache-2.0 (apify/crawlee), release v3.18.2. `npx crawlee create my-crawler` scaffolds a TypeScript project. Swap between a fast HTTP crawler and a full browser crawler with the same API, with browser fingerprinting and autoscaling built in. Runs anywhere; deploying to Apify is optional. ★~26k.","theCatch":"Needs Node 22.13 or newer. The Python version is a separate project with its own feature set, so do not assume parity. Heavy production crawling usually needs paid proxies to avoid blocks — the library rotates them, it does not supply them. Vs apify-mcp (gives an agent access to ready-made Apify scrapers) and playwright (the browser automation underneath): Crawlee is the crawler framework you build your own scraper on.","verdict":"The default Node scraping framework when a one-off Playwright script has turned into a crawler — budget for proxies.","addedAt":"2026-10-10","featured":false,"domain":"AI Agents","subSpecialty":"Web Scraping","subcategory":"Web Scraping","tags":["api","sdk","self-host","permissive","browser-automation","scraping","workflows","active"],"capabilities":["crawling","browser","scraping"],"surfaces":["API","Self-host"],"ecosystem":["Node.js"],"licenseModel":"Apache-2.0","githubStars":26188,"starsPerWeek":49,"access":"free","archived":false,"repo":{"createdAt":"2016-08-26","pushedAt":"2026-10-19","license":"Apache-2.0","archived":false,"checkedAt":"2026-10-19"}},"related":[{"slug":"hono","name":"Hono","tagline":"Tiny TypeScript web framework that runs on every runtime"},{"slug":"phaser","name":"Phaser","tagline":"MIT HTML5 2D game framework with scenes, physics, and tilemaps"},{"slug":"pixijs","name":"PixiJS","tagline":"MIT 2D WebGPU/WebGL renderer, not a game engine"},{"slug":"babylonjs","name":"Babylon.js","tagline":"Apache-2.0 3D engine for the web, now on v9"},{"slug":"playcanvas","name":"PlayCanvas","tagline":"MIT WebGL2/WebGPU engine plus an optional hosted Editor"}],"links":{"self":"/api/gems/crawlee","html":"/gems/crawlee","markdown":"/gems/crawlee.md","category":"/api/gems?category=libraries","categoryPage":"/stacks/libraries"}}