wpull
by ArchiveTeam
Wget-compatible web downloader and crawler.
AI summary
Website scraper
Downloads and crawls web pages, allowing for the archiving of websites.
- stars
- 556
- forks
- 77
- watching
- 23
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Web crawler
A web crawler designed to backup websites by recursively crawling and writing WARC files.
Archive extractor
A tool and package for extracting web archive data from popular sources like Wayback Machine and Common Crawl using the Go programming language.
Web crawler
A web crawler designed to efficiently collect and prioritize relevant content from the web
felipecsl/wombat1.3K
Web scraper library
A Ruby-based web crawler and data extraction tool with an elegant DSL.
Website crawler
Generates a sitemap of a website using Wayback Machine
s0rg/crawley268
Web URL extractor
A utility for systematically extracting URLs from web pages and printing them to the console.
Web Archiver
A graphical user interface layer for preserving and replaying web pages using multiple archiving tools.
crawler
Performs web page crawling at high performance.
Web Crawler
A distributed web crawler that fetches and extracts links from websites using a real browser.
Website archiver
A simple Rails application for archiving websites
WARC tools
Tools for working with archived web content
Content archiver
Automates archiving of online content from various sources into local storage or cloud services
Web archiver
A command-line tool for archiving and playing back websites in WARC format
Web crawler
A Node.js module for crawling web sites and scraping their content
Archiver
A tool to automate archiving of web resources into public archives.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions wpull.
