SkyScraper
Web scraper framework
A framework for building asynchronous web scrapers and crawlers using async/await and Reactive Extensions.
An asynchronous web scraper / web crawler using async / await and Reactive Extensions
59 stars
10 watching
18 forks
Language: C#
last commit: about 10 years agoLinked from 1 awesome list
brucedone/awesome-crawlerRelated projects:
| Repository | Description | Stars |
|---|---|---|
 crawlzone/crawlzone | A PHP framework for asynchronous internet crawling and web scraping | 78 |
 zhuyingda/webster | A framework for automating web scraping and crawling tasks using Node.js | 518 |
 slotix/dataflowkit | A framework for extracting structured data from web pages using CSS selectors. | 667 |
 howie6879/ruia | An async web scraping micro-framework built with asyncio and aiohttp to simplify URL crawling | 1,753 |
 carllerche/tower-web | A fast and asynchronous web framework built on top of Rust, removing boilerplate and decoupling HTTP concepts from application logic. | 980 |
 feng19/spider_man | A high-level web crawling and scraping framework for Elixir. | 23 |
 jcasts/gin | A lightweight Ruby web framework designed to support large applications with scalability and expressiveness. | 25 |
 codesofun/web-bee | A Java framework for building web-based crawlers with features like distributed crawling and proxy support. | 189 |
 untwisted/sukhoi | A minimalist web crawler framework built on top of miners and structure-based data extraction | 879 |
 joseconstela/webparsy | A Node.js library and CLI for scraping websites using Puppeteer and YAML definitions | 44 |
 antchfx/antch | A framework for building fast and efficient web crawlers and scrapers in Go. | 261 |
 sly777/ran | A full-stack React project using GraphQL and Next.js Toolkit for building fast and scalable web applications with built-in development tools. | 2,216 |
 jmg/crawley | A Pythonic framework for building high-speed web crawlers with flexible data extraction and storage options. | 188 |
 wspl/creeper | A framework for building cross-platform web crawlers using Go | 780 |
 ivanceras/sauron | A web framework and library that enables building client-side and server-side web applications with a simple, elegant, and Rust-based architecture. | 1,986 |