Crawl4AI

A crawler and scraper that turns websites into Markdown for retrieval systems and agents. It can run in your own Python process, a Docker server, or a hosted service.

Utility: Browser automation; kind: infrastructure.

unclecode/crawl4ai

Stars: 86k+; Starred in September: 4k+.

Star activity: September 2026.

Metrics updated: 31 Oct 2026. Profile reviewed: 31 Oct 2026.

Utilities

Who it is for

Requirements

Limitations

Cost and privacy notes

Illustrative example (untested)

Illustrative, not tested: crawl an allowlisted documentation section and inspect its Markdown before using it in retrieval.

License

Apache-2.0; reviewed LICENSE and README attribution guidance.

Primary sources