ache
Web crawler
A web crawler designed to efficiently collect and prioritize relevant content from the web
ACHE is a web crawler for domain-specific search.
454 stars
35 watching
135 forks
Language: Java
last commit: about 1 year ago
Linked from 1 awesome list
domain-specific-searchfocused-crawlerhacktoberfestweb-crawlerweb-scrapingweb-searchweb-spider
brucedone/awesome-crawlerRelated projects:
| Repository | Description | Stars |
|---|---|---|
 archiveteam/grab-site | A web crawler designed to backup websites by recursively crawling and writing WARC files. | 1,398 |
 amoilanen/js-crawler | A Node.js module for crawling web sites and scraping their content | 253 |
 uscdatascience/sparkler | A high-performance web crawler built on Apache Spark that fetches and analyzes web resources in real-time. | 410 |
| archiveteam/wpull | Downloads and crawls web pages, allowing for the archiving of websites. | 556 |
 stewartmckee/cobweb | A flexible web crawler that can be used to extract data from websites in a scalable and efficient manner | 226 |
 webrecorder/browsertrix-crawler | A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner. | 652 |
 fredwu/crawler | A high-performance web crawling and scraping solution with customizable settings and worker pooling. | 945 |
 helgeho/web2warc | A Web crawler that creates custom archives in WARC/CDX format | 24 |
 internetarchive/brozzler | A distributed web crawler that fetches and extracts links from websites using a real browser. | 671 |
 apache/incubator-stormcrawler | A collection of resources for building web crawlers on Apache Storm using Java | 891 |
 webis-de/wasp | A containerized web archive and search system using Elastic Search | 26 |
 spatie/laravel-site-search | A package to create a private search index by crawling and indexing a website | 274 |
 hominee/dyer | A fast and flexible web crawling tool with features like asynchronous I/O and event-driven design. | 133 |
 feng19/spider_man | A high-level web crawling and scraping framework for Elixir. | 23 |
 iamstoxe/urlgrab | A tool to crawl websites by exploring links recursively with support for JavaScript rendering. | 330 |