brozzler
brozzler - distributed browser-based web crawler
AI summary
Web Crawler
A distributed web crawler that fetches and extracts links from websites using a real browser.
- stars
- 678
- forks
- 99
- watching
- 39
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Crawler
A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner.
Web crawler
A web crawler designed to backup websites by recursively crawling and writing WARC files.
Web Crawler
A web crawler designed to crawl websites while obeying robots.txt rules, rate limits and concurrency limits, with customizable content handlers for parsing and processing crawled pages.
Web Crawler
A versatile web crawler built with modern tools and concurrency to handle various crawl tasks
Web Crawler
A flexible web crawler that can be used to extract data from websites in a scalable and efficient manner
hominee/dyer135
Web crawler
A fast and flexible web crawling tool with features like asynchronous I/O and event-driven design.
Crawler framework
A scalable and versatile web crawling framework based on Apache Storm
Web scraper
A Scala-based DSL for programmatically accessing and interacting with web pages
spider-rs/spider1.2K
Web scraper
A tool for web data extraction and processing using Rust
jmg/crawley188
Crawler
A Pythonic framework for building high-speed web crawlers with flexible data extraction and storage options.
Crawler
A Web crawler that creates custom archives in WARC/CDX format
Crawler
A concurrent web crawler written in Go that allows flexible and polite crawling of websites.
Web Crawler
A high-performance web crawler built on Apache Spark that fetches and analyzes web resources in real-time.
Crawler library
A Ruby web crawling library that provides flexible and customizable methods to crawl websites
Web crawler
A web crawler designed to efficiently collect and prioritize relevant content from the web
Featured in 1 awesome list
Each link jumps to the spot where the list mentions brozzler.
