SpiderSuite
Web Spider Crawler
A cross-platform web spider/crawler tool for analyzing and mapping attack surfaces
Advance web security spider/crawler
601 stars
10 watching
65 forks
last commit: about 1 year ago
Linked from 1 awesome list
bugbountycpluspluscrawlerguiinformation-gatheringosint-toolpentestqt5reconsecurity-toolsspiderweb-spiderwebcrawler
jivoi/awesome-osintRelated projects:
| Repository | Description | Stars |
|---|---|---|
 spider-rs/spider | A web crawler and scraper built on top of Rust, designed to extract data from the web in a flexible and configurable manner. | 1,140 |
 hu17889/go_spider | A modular, concurrent web crawler framework written in Go. | 1,826 |
 brendonboshell/supercrawler | A web crawler designed to crawl websites while obeying robots.txt rules, rate limits and concurrency limits, with customizable content handlers for parsing and processing crawled pages. | 378 |
 cocrawler/cocrawler | A versatile web crawler built with modern tools and concurrency to handle various crawl tasks | 187 |
 chenjiandongx/github-spider | A Python-based web crawler for scraping Github user and repository data. | 264 |
 feng19/spider_man | A high-level web crawling and scraping framework for Elixir. | 23 |
 turnersoftware/infinitycrawler | A web crawling library for .NET that allows customizable crawling and throttling of websites. | 248 |
 norconex/crawlers | A suite of tools for gathering and processing data from the web and file systems | 183 |
 mvdbos/php-spider | A flexible PHP web crawler with configurable traversal algorithms and filters. | 1,332 |
 iamstoxe/urlgrab | A tool to crawl websites by exploring links recursively with support for JavaScript rendering. | 330 |
 spiderlabs/malware-analysis | A collection of tools and scripts for analyzing malware | 244 |
 stewartmckee/cobweb | A flexible web crawler that can be used to extract data from websites in a scalable and efficient manner | 226 |
 amoilanen/js-crawler | A Node.js module for crawling web sites and scraping their content | 253 |
 postmodern/spidr | A Ruby web crawling library that provides flexible and customizable methods to crawl websites | 806 |
 webrecorder/browsertrix-crawler | A containerized browser-based crawler system for capturing web content in a high-fidelity and customizable manner. | 652 |