pipefunc
Workflow automator
Automates and simplifies the creation of function pipelines for efficient execution of scientific workflows.
Lightweight fast function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪
230 stars
2 watching
11 forks
Language: Python
last commit: 6 months ago
Linked from 1 awesome list
daghpcparallel-computingpipeline-frameworkpipelinesreproducible-researchslurmworkflow-engine
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 llnl/maestrowf | A tool to orchestrate computational workflows in high-performance computing environments. | 139 |
 pwwang/pipen | A Python-based workflow automation framework that enables easy creation of data processing pipelines | 105 |
 couler-proj/couler | Provides a unified interface for constructing and managing workflows across different workflow engines. | 919 |
 databiosphere/toil | A workflow management system designed to efficiently run pipelines in various environments. | 901 |
 scipipe/scipipe | A flexible and efficient way to write and run complex workflows using Go programming language | 1,081 |
 dagworks-inc/hamilton | Helps define and manage data transformations with a modular, self-documenting, and portable framework for directed acyclic graphs (DAGs) of data transformations. | 1,900 |
 symphony09/ograph | A framework for building data pipelines with concurrent execution and dependency management | 33 |
 paysure/orinoco | A functional composable pipeline framework for Python that separates business logic from implementation. | 11 |
 druths/xp | A tool for creating flexible and self-documenting data science pipelines | 56 |
 johnsonc/lambdo | A workflow engine for unifying feature engineering and machine learning operations in data analysis pipelines | 1 |
 moby/datakit | A tool to orchestrate applications using a version-controlled dataflow | 1,083 |
 it4innovations/hyperloom | A platform for defining and executing scientific pipelines in distributed environments using C++ and Python. | 16 |
 apache/airflow | A platform to programmatically author, schedule and monitor complex workflows | 37,580 |
 pharmbio/sciluigi | A lightweight wrapper around Spotify's Luigi workflow library to simplify writing scientific workflows | 335 |
 danielgerlag/liteflow | A Python library for running workflows with pluggable persistence and concurrency providers. | 62 |