crawler
by bplawler
Scala DSL for web crawling
AI summary
Web scraper
A Scala-based DSL for programmatically accessing and interacting with web pages
- stars
- 149
- forks
- 40
- watching
- 14
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
HTML parser
A Scala library providing a DSL for loading and extracting content from HTML pages
felipecsl/wombat1.3K
Web scraper library
A Ruby-based web crawler and data extraction tool with an elegant DSL.
Crawler library
A Ruby web crawling library that provides flexible and customizable methods to crawl websites
Web Crawler
A web crawler designed to crawl websites while obeying robots.txt rules, rate limits and concurrency limits, with customizable content handlers for parsing and processing crawled pages.
dyweb/scrala113
Web crawler framework
A web crawling framework written in Scala that allows users to define the start URL and parse response from it
Web Crawler
A distributed web crawler that fetches and extracts links from websites using a real browser.
Web scraper
A tool to extract data from web pages using various query languages and selectors.
Web Scraper
A web scraping library providing a declarative interface on top of an HTML parsing library to extract data from HTML pages
HTML scraper
A Perl toolkit for extracting structured data from HTML documents using a DSL-like interface.
Crawler
A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner.
URL detector
Detects and extracts URLs from text in written content
Web tester
A tool for end-to-end testing of web applications by crawling and comparing screenshots.
Web Crawler
A flexible web crawler that can be used to extract data from websites in a scalable and efficient manner
Website scraper
Downloads and crawls web pages, allowing for the archiving of websites.
Website scraper
A tool for scraping website content and analyzing browser behavior
Featured in 1 awesome list
Each link jumps to the spot where the list mentions crawler.
