Crawl4AI
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
Ranked by AgentPick Score, our data score from public GitHub activity. Description is the project's own.
Topics: ai, ai-agents, crawler, data-extraction, llm, markdown, mcp, open-source, playwright, python, rag, scraper
Starter prompt
Try Crawl4AI on one small, low-risk task: [task]. Tell me what worked, what didn't, and how long it took.
More in Agent infrastructure
DeepSeek Harness →
DeepSeek Harness: Everything is a Plugin.
Firecrawl →
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Netdata →
The fastest path to AI-powered full stack observability, even for lean teams.
OmniRoute →
Never stop coding. Free MIT AI gateway: one endpoint, 359 providers (150+ free), 1200+ models Kimi, Claude, GP…
CLIProxyAPI →
Wrap Antigravity, ChatGPT Codex, Claude Code, Grok Build, Muse Code, Devin as an OpenAI/Gemini/Claude/Codex co…
herdr →
the runtime your coding agents live on
AgentPick Weekly
The week's new agent tools with a verdict on each, every week. Free. We'll email you a link to confirm, and you can unsubscribe any time. See our Privacy Policy.