ache
Web crawler
A web crawler designed to efficiently collect and prioritize relevant content from the web
ACHE is a web crawler for domain-specific search.
459 stars
35 watching
134 forks
Language: Java
last commit: over 1 year ago
Linked from 1 awesome list
domain-specific-searchfocused-crawlerhacktoberfestweb-crawlerweb-scrapingweb-searchweb-spider
brucedone/awesome-crawlerRelated projects:
| Repository | Description | Stars |
|---|---|---|
 archiveteam/grab-site | A web crawler designed to backup websites by recursively crawling and writing WARC files. | 1,406 |
 amoilanen/js-crawler | A Node.js module for crawling web sites and scraping their content | 254 |
 uscdatascience/sparkler | A high-performance web crawler built on Apache Spark that fetches and analyzes web resources in real-time. | 411 |
| archiveteam/wpull | Downloads and crawls web pages, allowing for the archiving of websites. | 556 |
 stewartmckee/cobweb | A flexible web crawler that can be used to extract data from websites in a scalable and efficient manner | 226 |
 webrecorder/browsertrix-crawler | A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner. | 677 |
 fredwu/crawler | A high-performance web crawling and scraping solution with customizable settings and worker pooling. | 945 |
 helgeho/web2warc | A Web crawler that creates custom archives in WARC/CDX format | 25 |
 internetarchive/brozzler | A distributed web crawler that fetches and extracts links from websites using a real browser. | 678 |
 apache/incubator-stormcrawler | A scalable and versatile web crawling framework based on Apache Storm | 895 |
 webis-de/wasp | A containerized web archive and search system using Elastic Search | 27 |
 spatie/laravel-site-search | A package to create a private search index by crawling and indexing a website | 275 |
 hominee/dyer | A fast and flexible web crawling tool with features like asynchronous I/O and event-driven design. | 135 |
 feng19/spider_man | A high-level web crawling and scraping framework for Elixir. | 23 |
 iamstoxe/urlgrab | A tool to crawl websites by exploring links recursively with support for JavaScript rendering. | 331 |