webarchive-indexing
by ikreymer
Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
AI summary
WARC Indexer
Tools for bulk indexing of WARC/ARC files to create a shared url index
- stars
- 43
- forks
- 10
- watching
- 9
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
ARC file processor
Tool for partitioning and merging Web archive files by MIME type and year
Web archive indexer
Tools for indexing and discovering archived web content
WARC library
A fast streaming library for working with WARC format web archival data
WARC tools
Tools for working with archived web content
Web archiver
An HTTP proxy designed to capture and archive web traffic, including encrypted HTTPS connections.
Web archive parser
Provides tools for reading and parsing web archive formats used in digital preservation.
Web crawler
A web crawler designed to backup websites by recursively crawling and writing WARC files.
Web archiver
A web archiving tool that archives websites with high-fidelity preservation capabilities.
Converter
Converts HTTP Archive format to Web Archive format
Web archiver
A command-line tool for archiving and playing back websites in WARC format
WARC crawler
Converts HTTrack crawls to WARC files by reconstructing requests and responses from logs
WARC parser
A tool for parsing and generating Web Archive files in JavaScript using Node.js
chfoo/warcat152
Archive tool
Tool for handling Web Archive files
Web archive storage
A library for storing and querying web crawl data in a compact, easily sharable format.
Archiver
A tool for archiving webpages to IPFS
Featured in 1 awesome list
Each link jumps to the spot where the list mentions webarchive-indexing.
