heritrix3
Javapushed almost 2 years ago
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
AI summary
Crawler
A web crawler designed to collect and preserve digital artifacts while respecting site policies and load constraints.
- stars
- 2.9K
- forks
- 762
- watching
- 187
- awesome lists
- 3
Featured in 3 awesome lists
Each link jumps to the spot where the list mentions heritrix3.