crawl4ai
by unclecode
🚀🤖 Crawl4AI: Crawl Smarter, Faster, Freely. For AI.
AI summary
Web crawler
A web crawling tool designed to extract structured data from the web for use in AI applications
- stars
- 18.5K
- forks
- 1.4K
- watching
- 109
Similar projects
Found by comparing what the projects do, not just their names.
code4craft/webmagic11.5K
crawler
A framework for building scalable web crawlers in Java.
apify/crawlee16.1K
Web scraper
A tool for building reliable web scraping and browser automation pipelines in Node.js.
spatie/crawler2.6K
Crawler
A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently.
gocolly/colly23.4K
Website scraper
A framework for extracting structured data from websites in a fast and elegant way
Web crawler
A Java-based web crawler for extracting and processing web page content
Crawler
A distributed crawling framework that leverages Headless Chrome to scrape dynamic websites
s0md3v/photon11.1K
Crawler
A fast and flexible web crawler designed to gather information from the internet
Web scraper
A NodeJS-based web crawler and spider that extracts data from websites.
s0rg/crawley268
Web URL extractor
A utility for systematically extracting URLs from web pages and printing them to the console.
scraper
A Node.js library and CLI tool for automating web page scraping and parsing
howie6879/ruia1.8K
Crawler framework
An async web scraping micro-framework built with asyncio and aiohttp to simplify URL crawling
Crawler
A web crawler designed to collect and preserve digital artifacts while respecting site policies and load constraints.
crawler
A Python web crawling framework utilizing asyncio and aiohttp for efficient data extraction from websites.
Web Crawler
A flexible web crawler that can be used to extract data from websites in a scalable and efficient manner
Web crawler
A web crawler designed to backup websites by recursively crawling and writing WARC files.
