warctools
Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
AI summary
WARC tools
Tools for working with archived web content
- stars
- 153
- forks
- 28
- watching
- 44
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
chfoo/warcat152
Archive tool
Tool for handling Web Archive files
Web archiver
An HTTP proxy designed to capture and archive web traffic, including encrypted HTTPS connections.
WARC library
A fast streaming library for working with WARC format web archival data
Converter
Converts HTTP Archive format to Web Archive format
WARC crawler
Converts HTTrack crawls to WARC files by reconstructing requests and responses from logs
Web crawler
A web crawler designed to backup websites by recursively crawling and writing WARC files.
Web archiver
A command-line tool for archiving and playing back websites in WARC format
WARC Indexer
Tools for bulk indexing of WARC/ARC files to create a shared url index
Web archive indexer
Tools for indexing and discovering archived web content
Web archiver
A web archiving tool that archives websites with high-fidelity preservation capabilities.
Web archive parser
Provides tools for reading and parsing web archive formats used in digital preservation.
ARC file processor
Tool for partitioning and merging Web archive files by MIME type and year
WARC library
A Java library for reading and writing WARC files with a typed API
Website scraper
Downloads and crawls web pages, allowing for the archiving of websites.
WARC tour
Tours using Common Crawl's WARC format data to demonstrate its structure and contents
Featured in 1 awesome list
Each link jumps to the spot where the list mentions warctools.