mdf

Data-flow programming engine

A toolkit for expressing programs as directed acyclic graphs and wiring together computations over time-series data.

Data-flow programming toolkit for Python

GitHub

169 stars
29 watching
52 forks
Language: Python
last commit: about 8 years ago

Related projects:

Repository Description Stars
nipype/pydra A lightweight Python dataflow engine for building and executing directed acyclic graphs (DAGs) in a scalable manner. 123
dagworks-inc/hamilton Helps define and manage data transformations with a modular, self-documenting, and portable framework for directed acyclic graphs (DAGs) of data transformations. 1,900
mhkit-software/mhkit-python A Python package for marine renewable energy data processing and visualization 50
mroberge/hydrofunctions A suite of convenience functions for exploring and visualizing hydrology data 66
mehd-io/pypi-duck-flow A data engineering project that extracts insights from Python projects using DuckDB and MotherDuck. 173
eclipse-zenoh-flow/zenoh-flow A framework for declarative data flow programming and edge computing 92
johnsonc/lambdo A workflow engine for unifying feature engineering and machine learning operations in data analysis pipelines 1
usepa/fedelemflowlist Generates and provides standardized elementary flow lists for life cycle assessment data conversion 34
d6t/d6tflow A Python library to build and manage complex data science workflows efficiently 953
deqitang/pymatflow Automates preparation and submission of DFT calculations for materials science simulations 6
giacbrd/smartpipeline A framework for designing and executing concurrent data pipelines with a focus on simplicity and efficiency 25
log2timeline/dftimewolf A framework for orchestrating data collection, processing, and export 299
secure-software-engineering/flowdroid Analyzes data flows in software to identify potential security vulnerabilities and weaknesses. 1,076
streamlet-dev/tributary A Python library for constructing dataflow graphs with support for reactive and lazy evaluation. 444
huggingface/datatrove A platform-agnostic data processing framework for large-scale text data pipelines 2,103