gogetcrawl
by karust
Extract web archive data using Wayback Machine and Common Crawl
AI summary
Archive extractor
A tool and package for extracting web archive data from popular sources like Wayback Machine and Common Crawl using the Go programming language.
- stars
- 148
- forks
- 17
- watching
- 5
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
s0rg/crawley268
Web URL extractor
A utility for systematically extracting URLs from web pages and printing them to the console.
Data extractor
A collection of tools for extracting and analyzing data from web archives
URL extractor
A tool to extract URLs from HTML attributes by crawling in and evaluating JavaScript
Website scraper
Downloads and crawls web pages, allowing for the archiving of websites.
Web archiver
Archives a web page as a single HTML file with embedded resources.
Web crawler
A web crawler designed to backup websites by recursively crawling and writing WARC files.
Archive extractor
A .NET implementation of the Atom Asar archive format, allowing extraction and manipulation of archived files.
Keyword extractor
An algorithm for extracting keywords from text based on word frequency and part-of-speech tagging
Web archive parser
Provides tools for reading and parsing web archive formats used in digital preservation.
wabarc/wayback1.8K
Web archiver
A tool for capturing and preserving web content and making it accessible in the future.
JSON extractor
A library to extract specific properties from complex JSON structures into Go structs with minimal code changes.
Archiver
A tool to automate archiving of web resources into public archives.
Archive extractor
A plugin that allows users to extract files from various archive formats without specifying the extraction command.
Web archiver
A tool for archiving and preserving online content, allowing users to salvage and store websites, articles, contributions, text, and documentation.
Git extractor
A tool for extracting specific files from git repositories during penetration testing without downloading the entire repository.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions gogetcrawl.
