mdf
Data-flow programming engine
A toolkit for expressing programs as directed acyclic graphs and wiring together computations over time-series data.
Data-flow programming toolkit for Python
169 stars
29 watching
52 forks
Language: Python
last commit: about 8 years ago Related projects:
Repository | Description | Stars |
---|---|---|
nipype/pydra | A lightweight Python dataflow engine for building and executing directed acyclic graphs (DAGs) in a scalable manner. | 123 |
dagworks-inc/hamilton | Helps define and manage data transformations with a modular, self-documenting, and portable framework for directed acyclic graphs (DAGs) of data transformations. | 1,900 |
mhkit-software/mhkit-python | A Python package for marine renewable energy data processing and visualization | 50 |
mroberge/hydrofunctions | A suite of convenience functions for exploring and visualizing hydrology data | 66 |
mehd-io/pypi-duck-flow | A data engineering project that extracts insights from Python projects using DuckDB and MotherDuck. | 173 |
eclipse-zenoh-flow/zenoh-flow | A framework for declarative data flow programming and edge computing | 92 |
johnsonc/lambdo | A workflow engine for unifying feature engineering and machine learning operations in data analysis pipelines | 1 |
usepa/fedelemflowlist | Generates and provides standardized elementary flow lists for life cycle assessment data conversion | 34 |
d6t/d6tflow | A Python library to build and manage complex data science workflows efficiently | 953 |
deqitang/pymatflow | Automates preparation and submission of DFT calculations for materials science simulations | 6 |
giacbrd/smartpipeline | A framework for designing and executing concurrent data pipelines with a focus on simplicity and efficiency | 25 |
log2timeline/dftimewolf | A framework for orchestrating data collection, processing, and export | 299 |
secure-software-engineering/flowdroid | Analyzes data flows in software to identify potential security vulnerabilities and weaknesses. | 1,076 |
streamlet-dev/tributary | A Python library for constructing dataflow graphs with support for reactive and lazy evaluation. | 444 |
huggingface/datatrove | A platform-agnostic data processing framework for large-scale text data pipelines | 2,103 |