anemone

Spider

A web spider framework that crawls domains and collects information about visited pages

Anemone web-spider framework

GitHub
2k stars
63 watching
324 forks
Language: Ruby
last commit: over 4 years ago
Linked from 1 awesome list


Screenshot of chriskite/anemone website

anemone.rubyforge.org

Backlinks from these awesome lists:

Related projects:

Repository Description Stars
chrispenner/firefly A simple Haskell web framework for building lightweight servers with monad-based request handling. 87
crepe/crepe A lightweight Ruby framework for building fast and flexible APIs 128
stephencelis/crepe A Ruby-based thin API stack for building Twitter APIs 2
spider-gazelle/spider-gazelle A web framework template built on top of Crystal that aims to provide speed and extensibility for building Ruby-like web applications. 180
arachne-framework/aristotle Provides a Clojure interface to RDF and OWL libraries for data modeling and querying 130
hu17889/go_spider A modular, concurrent web crawler framework written in Go. 1,826
kballenegger/kenji A lightweight Ruby web framework for building backend APIs with clean routing and modular code organization. 32
3nock/spidersuite A cross-platform web spider/crawler tool for analyzing and mapping attack surfaces 601
feng19/spider_man A high-level web crawling and scraping framework for Elixir. 23
dyweb/scrala A web crawling framework written in Scala that allows users to define the start URL and parse response from it 113
postmodern/spidr A Ruby web crawling library that provides flexible and customizable methods to crawl websites 806
jcasts/gin A lightweight Ruby web framework designed to support large applications with scalability and expressiveness. 25
elixir-crawly/crawly A framework for extracting structured data from websites 987
colinkiama/spider A tool for quickly generating a basic HTML5 site structure 2
puerkitobio/gocrawl A concurrent web crawler written in Go that allows flexible and polite crawling of websites. 2,038