dbnd
Pipeline manager
A framework for building and tracking data pipelines to simplify data engineering workflows
DBND is an agile pipeline framework that helps data engineering teams track and orchestrate their data processes.
251 stars
16 watching
35 forks
Language: Python
last commit: 21 days ago
Linked from 1 awesome list
ethicalml/awesome-production-machine-learningRelated projects:
| Repository | Description | Stars |
|---|---|---|
 dagster-io/dagster | An orchestration platform for data pipelines and assets, providing a declarative programming model and integrated lineage and observability. | 11,699 |
 jazzband/django-pipeline | An asset packaging library for Django that simplifies CSS and JavaScript concatenation and compression. | 1,517 |
 moby/datakit | A tool to orchestrate applications using a version-controlled dataflow | 1,082 |
 apache/airflow | A platform to programmatically author, schedule and monitor complex workflows | 37,120 |
 bl3f/yato | An orchestrator for DuckDB databases that automates data transformation and integration with other tools. | 174 |
 algorand/conduit | A framework for ingesting blocks from a blockchain into external applications | 37 |
 log2timeline/dftimewolf | A framework for orchestrating data collection, processing, and export | 296 |
| apache/streampipes | A toolbox for industrial data analytics and stream processing | 605 |
 huggingface/datatrove | A platform-agnostic data processing framework for large-scale text data pipelines | 2,043 |
 giacbrd/smartpipeline | A framework for designing and executing concurrent data pipelines with a focus on simplicity and efficiency | 23 |
 alan-turing-institute/xpandas | A high-level data container with transformer functionality for feature extraction and transformation. | 26 |
 johnsonc/lambdo | A workflow engine for unifying feature engineering and machine learning operations in data analysis pipelines | 1 |
 kevin-hanselman/dud | A lightweight tool for managing and versioning large data alongside source code in data pipelines | 183 |
 d6t/d6tflow | A Python library to build and manage complex data science workflows efficiently | 952 |
 pipefunc/pipefunc | Automates and simplifies the creation of function pipelines for efficient execution of scientific workflows. | 215 |