grab
Web scraper
A Python framework for asynchronous web scraping and crawling with a flexible network backend.
Web Scraping Framework
2k stars
89 watching
275 forks
Language: Python
last commit: 8 months ago
Linked from 2 awesome lists
asynchronouscrawlercrawlingframeworkhttp-clientnetworkpycurlpythonpython-librarypython3scrapingspiderurllib3web-scraping
vinta/awesome-python
lichunqiang/awesome-starsRelated projects:
| Repository | Description | Stars |
|---|---|---|
 ruipgil/scraperjs | A versatile web scraping module with two scrapers for static and dynamic content extraction. | 3,710 |
 gocolly/colly | A framework for extracting structured data from websites in a fast and elegant way | 23,317 |
 apify/crawlee | A tool for building reliable web scraping and browser automation pipelines in Node.js. | 15,604 |
 matthewmueller/x-ray | A flexible web scraping framework for extracting data from websites with customizable selectors and pagination support. | 5,878 |
 hugapi/hug | A Python framework for building simple and efficient APIs | 6,862 |
 ionicabizau/scrape-it | A Node.js library and CLI tool for automating web page scraping and parsing | 4,012 |
 unclecode/crawl4ai | A tool for web crawling and data extraction, designed to work with large language models. | 16,180 |
 apiaryio/dredd | Tool for validating API implementations against their own documentation | 4,194 |
 segment-boneyard/nightmare | A high-level browser automation library that allows users to interact with web pages in a synchronous manner. | 19,548 |
 clips/pattern | A comprehensive Python module for web mining and analysis of text data. | 8,750 |
 malwaredllc/byob | An open-source framework for creating custom post-exploitation tools with automated payload generation and platform independence. | 8,989 |
 spatie/crawler | A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently. | 2,537 |
 cobrateam/splinter | A Python test framework for automating web applications using Selenium and other tools. | 2,722 |
 manisso/fsociety | A comprehensive collection of hacking tools and scripts for penetration testing and vulnerability assessment | 10,637 |
 marionettejs/backbone.marionette | A composite application library for Backbone.js that simplifies the construction of large-scale JavaScript applications | 7,061 |