kimuraframework
Scraping framework
A web scraping framework for extracting data from JavaScript-rendered websites.
Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites
1k stars
30 watching
155 forks
Language: Ruby
last commit: 6 months ago
Linked from 1 awesome list
crawlerheadless-chromekimuraiscraperscrapy
markets/awesome-rubyRelated projects:
| Repository | Description | Stars |
|---|---|---|
 zhuyingda/webster | A framework for automating web scraping and crawling tasks using Node.js | 515 |
 matiasb/demiurge | An HTML scraping framework built on top of PyQuery. | 114 |
 wardrop/scorched | A light-weight web framework for Ruby that aims to provide generic yet powerful constructs for processing HTTP requests. | 275 |
 joncanning/skyscraper | A framework for building asynchronous web scrapers and crawlers using async/await and Reactive Extensions. | 58 |
 slotix/dataflowkit | A framework for extracting structured data from web pages using CSS selectors. | 662 |
 kballenegger/kenji | A lightweight Ruby web framework for building backend APIs with clean routing and modular code organization. | 32 |
 kenichi/angelo | A DSL for building web applications with real-time capabilities using Ruby and Reel's reactor | 302 |
 guilleiguaran/nancy | A Ruby web framework that provides a minimalistic approach to building web applications with features like routing, session management and template rendering. | 82 |
 joseconstela/webparsy | A Node.js library and CLI for scraping websites using Puppeteer and YAML definitions | 44 |
 propublica/upton | A web scraping framework that simplifies the process by handling repetitive tasks and provides options for efficient data retrieval | 1,613 |
 jjelosua/doga_scraper | A tool that extracts and converts Galician Official journal documents to different formats based on input year. | 0 |
 komarserjio/notejam | A unified sample web application to facilitate learning of various server-side frameworks. | 1,145 |
 miyagawa/web-scraper | A Perl toolkit for extracting structured data from HTML documents using a DSL-like interface. | 104 |
 kimeiga/bahunya | A CSS framework that uses semantic HTML to provide a set of pre-defined UI elements without using explicit class names. | 297 |
 oscarotero/embed | A PHP library to extract metadata and embeddable code from any web page using various protocols and scraping techniques. | 2,091 |