InfinityCrawler
A simple but powerful web crawler library for .NET
AI summary
Crawler library
A web crawling library for .NET that allows customizable crawling and throttling of websites.
- stars
- 248
- forks
- 36
- watching
- 11
- awesome lists
- 3
Similar projects
Found by comparing what the projects do, not just their names.
Web Crawler
A web crawler designed to crawl websites while obeying robots.txt rules, rate limits and concurrency limits, with customizable content handlers for parsing and processing crawled pages.
Crawler library
A Rust-based library for building and managing cryptocurrency crawlers
Web Crawler
A versatile web crawler built with modern tools and concurrency to handle various crawl tasks
Crawler
A concurrent web crawler written in Go that allows flexible and polite crawling of websites.
Crawler
A distributed web crawler that coordinates crawling tasks across multiple worker processes using a message bus.
Crawler framework
A distributed crawler framework that simplifies the process of building crawlers using Spring Boot and Redis
Crawler library
A Ruby web crawling library that provides flexible and customizable methods to crawl websites
Crawler library
A high-level web crawling and scraping framework for Elixir.
Crawler
A modular, concurrent web crawler framework written in Go.
Crawler
A high-performance web crawling and scraping solution with customizable settings and worker pooling.
Crawler
A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner.
Web crawler
A Node.js module for crawling web sites and scraping their content
wspl/creeper780
Crawler framework
A framework for building cross-platform web crawlers using Go
Presentation library
A .NET library for creating and manipulating PowerPoint presentations using Open XML.
qinxuye/cola1.5K
Crawler library
A high-level framework for building distributed data extractors from web pages
Featured in 3 awesome lists
Each link jumps to the spot where the list mentions InfinityCrawler.
