d6tflow

Data workflow builder

A Python library to build and manage complex data science workflows efficiently

Python library for building highly effective data science workflows

GitHub

952 stars
24 watching
77 forks
Language: Python
last commit: over 1 year ago
Linked from 1 awesome list


Backlinks from these awesome lists:

Related projects:

Repository Description Stars
danielgerlag/liteflow A Python library for running workflows with pluggable persistence and concurrency providers. 61
deqitang/pymatflow Automates preparation and submission of DFT calculations for materials science simulations 5
gwforg/gwf A tool for building and running large scientific workflows using Python 3.7+ 31
druths/xp A tool for creating flexible and self-documenting data science pipelines 56
dataoneorg/d1_python A collection of Python libraries and tools for interacting with DataONE repositories 17
pveber/bistro A library to build and execute complex, distributed workflows in a declarative manner 47
whitaker-io/machine A library for creating data workflows that can be simple or complex, with features like recursion and memoization. 158
databiosphere/toil A workflow management system designed to efficiently run pipelines in various environments. 901
astrazeneca/runnable A Python framework for defining and executing data science workflows with modular components 40
pydoit/doit An automation tool that enables efficient task management and execution of complex workflows 1,871
nvidia/earth2studio A Python-based framework for building and deploying AI-driven weather and climate workflows 98
baffelli/pyperator A Python library for building asynchronous workflows with a directed acyclic graph structure 60
d3xterjs/pydexter A Python wrapper for rendering charts in the browser with simple syntax 31
log2timeline/dftimewolf A framework for orchestrating data collection, processing, and export 296
nipype/pydra A lightweight Python dataflow engine for building and executing directed acyclic graphs (DAGs) in a scalable manner. 120