grab
Web scraper
A Python framework for asynchronous web scraping and crawling with a flexible network backend.
Web Scraping Framework
2k stars
89 watching
274 forks
Language: Python
last commit: 12 months ago
Linked from 2 awesome lists
asynchronouscrawlercrawlingframeworkhttp-clientnetworkpycurlpythonpython-librarypython3scrapingspiderurllib3web-scraping
vinta/awesome-python
lichunqiang/awesome-starsRelated projects:
| Repository | Description | Stars |
|---|---|---|
 ruipgil/scraperjs | A versatile web scraping module with two scrapers for static and dynamic content extraction. | 3,714 |
 gocolly/colly | A framework for extracting structured data from websites in a fast and elegant way | 23,444 |
 apify/crawlee | A tool for building reliable web scraping and browser automation pipelines in Node.js. | 16,081 |
 matthewmueller/x-ray | A flexible web scraping framework for extracting data from websites with customizable selectors and pagination support. | 5,883 |
 hugapi/hug | A Python framework for building simple and efficient APIs | 6,864 |
 ionicabizau/scrape-it | A Node.js library and CLI tool for automating web page scraping and parsing | 4,024 |
 unclecode/crawl4ai | A web crawling tool designed to extract structured data from the web for use in AI applications | 18,541 |
 apiaryio/dredd | Tool for validating API implementations against their own documentation | 4,192 |
 segment-boneyard/nightmare | A high-level browser automation library that allows users to interact with web pages in a synchronous manner. | 19,555 |
 clips/pattern | A comprehensive Python module for web mining and analysis of text data. | 8,758 |
 malwaredllc/byob | An open-source framework for creating custom post-exploitation tools with automated payload generation and platform independence. | 9,005 |
 spatie/crawler | A powerful web crawler written in PHP that can execute JavaScript and crawl multiple URLs concurrently. | 2,552 |
 cobrateam/splinter | A Python test framework for automating web applications using Selenium and other tools. | 2,726 |
 manisso/fsociety | A comprehensive collection of hacking tools and scripts for penetration testing and vulnerability assessment | 10,698 |
 marionettejs/backbone.marionette | A composite application library for Backbone.js that simplifies the construction of large-scale JavaScript applications | 7,057 |