hakrawler
by hakluke
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
AI summary
Crawler
A tool for automatically discovering and crawling web application endpoints and assets
- stars
- 4.5K
- forks
- 498
- watching
- 63
- awesome lists
- 2
Similar projects
Found by comparing what the projects do, not just their names.
Crawler
A tool for web crawling and exploitation written in Go.
Link crawler
A tool to crawl websites by exploring links recursively with support for JavaScript rendering.
s0rg/crawley268
Web URL extractor
A utility for systematically extracting URLs from web pages and printing them to the console.
geziyor/geziyor2.6K
crawler
A fast and flexible web crawling and scraping framework for extracting structured data from websites.
Domain scanner
A Golang client for querying SecurityTrails API data to gather information on domains and their associated networks.
Web crawler
A Java-based web crawler for extracting and processing web page content
gocolly/colly23.4K
Website scraper
A framework for extracting structured data from websites in a fast and elegant way
s0md3v/photon11.1K
Crawler
A fast and flexible web crawler designed to gather information from the internet
spatie/crawler2.6K
Crawler
A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently.
Crawler
A distributed crawling framework that leverages Headless Chrome to scrape dynamic websites
apify/crawlee16.1K
Web scraper
A tool for building reliable web scraping and browser automation pipelines in Node.js.
unclecode/crawl4ai18.5K
Web crawler
A web crawling tool designed to extract structured data from the web for use in AI applications
Web Crawler
A web crawler designed to crawl websites while obeying robots.txt rules, rate limits and concurrency limits, with customizable content handlers for parsing and processing crawled pages.
Crawler
A concurrent web crawler written in Go that allows flexible and polite crawling of websites.
code4craft/webmagic11.5K
crawler
A framework for building scalable web crawlers in Java.
Featured in 2 awesome lists
Each link jumps to the spot where the list mentions hakrawler.
