scrapy-redis
Crawler helper
Provides Redis-based components for distributed crawling and post-processing in Scrapy
Redis-based components for Scrapy.
6k stars
273 watching
2k forks
Language: Python
last commit: 5 months ago
Linked from 1 awesome list
crawlerdistributedredisscrapy
brucedone/awesome-crawlerRelated projects:
| Repository | Description | Stars |
|---|---|---|
 anjia0532/lua-resty-redis-util | A Lua utility library providing a simplified interface to the Lua-resty-redis library | 124 |
 openresty/xss-nginx-module | Enables cross-site scripting support in an Nginx configuration to facilitate secure AJAX responses. | 148 |
 lnlyssg/zos | Provides tools and utilities for working with RACF and z/OS security settings on mainframes | 33 |
 stackexchange/stackexchange.redis | A high-level interface to the Redis in-memory data store | 5,911 |
 eddyystop/feathers-starter-react-redux-login-roles | A starter package for building commercial applications with Feathers and Redux | 195 |
| openresty/lua-redis-parser | A Lua module that parses raw Redis responses and constructs corresponding data structures. | 93 |
 taherio/redi | Automates setup of CobaltStrike redirectors using nginx reverse proxy and LetsEncrypt SSL certificates | 140 |
| openresty/lua-resty-redis | Provides a Lua interface to Redis, leveraging the nonblocking cosocket API for high-performance connectivity. | 1,904 |
 ledgetech/lua-resty-redis-connector | A Lua module for establishing connections to Redis servers with various authentication options and configurations. | 235 |
 rschroll/rmfuse | Provides access to reMarkable Cloud files through FUSE filesystem | 95 |
 redis/redis-rb | A Ruby client library for interacting with the Redis in-memory data store | 3,976 |
 nateware/redis-objects | A Ruby interface to Redis that maps data types to Ruby objects, enabling atomic operations and simplifying integration with existing ORMs. | 2,090 |
 aembke/fred.rs | An async Redis client with various features and protocols for interacting with Redis servers. | 390 |
 jagracey/regex-dos | Scans Regular Expressions for potential performance issues that can be exploited for Denial of Service attacks | 162 |
 steve0511/resty-redis-cluster | A Lua client module for interacting with Redis clusters using OpenResty. | 376 |