crawly

Spider

A framework for extracting structured data from websites

Crawly, a high-level web crawling & scraping framework for Elixir.

GitHub
987 stars
20 watching
116 forks
Language: Elixir
last commit: 2 months ago
Linked from 1 awesome list

crawlercrawlingelixirerlangextract-datascraperscrapingscraping-websitesspider

Screenshot of elixir-crawly/crawly website

hexdocs.pm/crawly

Backlinks from these awesome lists:

Related projects:

Repository Description Stars
feng19/spider_man A high-level web crawling and scraping framework for Elixir. 23
anonyfox/elixir-scrape A tool for extracting structured data from web resources using information-retrieval techniques. 328
fredwu/crawler A high-performance web crawling and scraping solution with customizable settings and worker pooling. 945
spider-rs/spider A web crawler and scraper built on top of Rust, designed to extract data from the web in a flexible and configurable manner. 1,140
hu17889/go_spider A modular, concurrent web crawler framework written in Go. 1,826
gushonorato/mechanize A web scraping and automation tool for Elixir. 30
pigmej/exelli An Elixir wrapper with a simple syntax for building web applications. 16
junekelly/sneeze Tools for rendering Elixir data structures into HTML 59
antchfx/antch A framework for building fast and efficient web crawlers and scrapers in Go. 260
dyweb/scrala A web crawling framework written in Scala that allows users to define the start URL and parse response from it 113
howie6879/ruia An async web scraping micro-framework built with asyncio and aiohttp to simplify URL crawling 1,752
elixir-soap/soap A Elixir library to interact with SOAP web services 135
postmodern/spidr A Ruby web crawling library that provides flexible and customizable methods to crawl websites 806
chenjiandongx/github-spider A Python-based web crawler for scraping Github user and repository data. 264
rdf-elixir/sparql-ex An implementation of the SPARQL query language for Elixir to execute queries against RDF data structures 38