scrapy-mongodb
by sebdah
MongoDB pipeline for Scrapy. This module supports both MongoDB in standalone setups and replica sets. scrapy-mongodb will insert the items to MongoDB as soon as your spider finds data to extract.
AI summary
Data pipeline
A MongoDB pipeline extension for Scrapy spiders that enables real-time data insertion and buffering options.
- stars
- 357
- forks
- 99
- watching
- 26
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Data pipeline tool
Enables real-time data synchronization between MongoDB and other systems.
MongoDB interface
Provides an object-oriented interface to MongoDB
MongoDB client
A Python library for interacting with MongoDB using object-document mapping and asynchronous support
Mongo DB mapper
A Python library providing an object-document mapper for MongoDB with support for synchronous and asynchronous operations.
MongoDB wrapper
Provides a way to interact with MongoDB databases using Emacs Lisp
Data source wrapper
An extension for DuckDB that allows running SQL queries on arbitrary data sources using Python functions.
Command explainer
Provides a simplified interface to explain MongoDB commands in PyMongo
DB substitute
A pure Python-implemented alternative to MongoDB.
Data pipeline
A data engineering project that extracts insights from Python projects using DuckDB and MotherDuck.
Spider manager
An app that allows you to manage Scrapy spiders through a Django admin interface.
MongoDB wrapper
A PostgreSQL extension that enables interaction with MongoDB databases through foreign data wrappers.
Data cleaner
Library that provides dynamic data cleaning and filtering capabilities for PyTorch datasets and samplers
MongoDB replicator
A tool for setting up multi-master replication with MongoDB
Database connector
Provides a bridge between Django and MongoDB databases
MongoDB client
A RESTful API for interacting with MongoDB databases
Featured in 1 awesome list
Each link jumps to the spot where the list mentions scrapy-mongodb.
