crawler
by fredwu
A high performance web crawler / scraper in Elixir.
AI summary
Crawler
A high-performance web crawling and scraping solution with customizable settings and worker pooling.
- stars
- 945
- forks
- 91
- watching
- 32
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Spider
A framework for extracting structured data from websites
Crawler library
A high-level web crawling and scraping framework for Elixir.
Crawler
A distributed web crawler that coordinates crawling tasks across multiple worker processes using a message bus.
spider-rs/spider1.2K
Web scraper
A tool for web data extraction and processing using Rust
Crawler
A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner.
Crawler
A modular, concurrent web crawler framework written in Go.
Web crawler
A web crawler designed to efficiently collect and prioritize relevant content from the web
Crawler library
A web crawling library for .NET that allows customizable crawling and throttling of websites.
crawler
A Python web crawling framework utilizing asyncio and aiohttp for efficient data extraction from websites.
Crawler
A concurrent web crawler written in Go that allows flexible and polite crawling of websites.
Crawler
A Python-based web crawler for scraping Github user and repository data.
crawler
Performs web page crawling at high performance.
Crawler
A flexible web crawler that follows robots.txt policies and crawl delays.
Web crawler
A framework for building fast and efficient web crawlers and scrapers in Go.
Crawler
A Web crawler that creates custom archives in WARC/CDX format
Featured in 1 awesome list
Each link jumps to the spot where the list mentions crawler.
