xidel
Web scraper
A tool to extract data from web pages using various query languages and selectors.
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
690 stars
27 watching
42 forks
Language: Pascal
last commit: over 2 years agoLinked from 1 awesome list
clicommand-linecss-selectorcurldata-processingdatascrapinghtmlhttphttpiejsonrestscraperwebwebscraperwebscrapingwgetxmlxmlstarletxpathxquery
alebcay/awesome-shellRelated projects:
| Repository | Description | Stars |
|---|---|---|
 felipecsl/wombat | A Ruby-based web crawler and data extraction tool with an elegant DSL. | 1,315 |
 the-markup/blacklight-collector | A tool for scraping website content and analyzing browser behavior | 205 |
 miyagawa/web-scraper | A Perl toolkit for extracting structured data from HTML documents using a DSL-like interface. | 104 |
 joseconstela/webparsy | A Node.js library and CLI for scraping websites using Puppeteer and YAML definitions | 44 |
 spekulatius/phpscraper | A web scraping utility for PHP that simplifies the process of extracting information from websites. | 544 |
 slotix/dataflowkit | A framework for extracting structured data from web pages using CSS selectors. | 667 |
 medialab/minet | A command line tool and Python library for extracting data from various web sources. | 293 |
 jaimeiniesta/metainspector | A Ruby gem for web scraping and extracting metadata from web pages. | 1,038 |
 bplawler/crawler | A Scala-based DSL for programmatically accessing and interacting with web pages | 149 |
 oscarotero/embed | A PHP library to retrieve metadata and embed code from any web page | 2,100 |
 zhuyingda/webster | A framework for automating web scraping and crawling tasks using Node.js | 518 |
 spider-rs/spider | A tool for web data extraction and processing using Rust | 1,234 |
 gushonorato/mechanize | A web scraping and automation tool for Elixir. | 30 |
 meilisearch/docs-scraper | Automates scraping and indexing of documentation content into a search engine | 297 |
 jakopako/goskyr | A tool to simplify web scraping of list-like structured data from web pages | 36 |