owlcrawler
by fmpwizard
Crawl the web using nats.io and Go
AI summary
Crawler
A distributed web crawler that coordinates crawling tasks across multiple worker processes using a message bus.
- stars
- 55
- forks
- 4
- watching
- 9
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Crawler
A concurrent web crawler written in Go that allows flexible and polite crawling of websites.
Crawler
A high-performance web crawling and scraping solution with customizable settings and worker pooling.
Crawler
A modular, concurrent web crawler framework written in Go.
Web Crawler
A web crawler designed to crawl websites while obeying robots.txt rules, rate limits and concurrency limits, with customizable content handlers for parsing and processing crawled pages.
Web Crawler
A versatile web crawler built with modern tools and concurrency to handle various crawl tasks
Crawler library
A web crawling library for .NET that allows customizable crawling and throttling of websites.
wspl/creeper780
Crawler framework
A framework for building cross-platform web crawlers using Go
Crawler
A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner.
Web crawler
A framework for building fast and efficient web crawlers and scrapers in Go.
Crawler
A flexible web crawler that follows robots.txt policies and crawl delays.
jmg/crawley188
Crawler
A Pythonic framework for building high-speed web crawlers with flexible data extraction and storage options.
Web Crawler
A distributed web crawler that fetches and extracts links from websites using a real browser.
Crawler framework
A scalable and versatile web crawling framework based on Apache Storm
Crawler framework
A distributed crawler framework that simplifies the process of building crawlers using Spring Boot and Redis
Web Crawler
A flexible PHP web crawler with configurable traversal algorithms and filters.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions owlcrawler.
