toil
Pipeline runner
A workflow management system designed to efficiently run pipelines in various environments.
A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.
900 stars
56 watching
240 forks
Language: Python
last commit: 6 days ago
Linked from 1 awesome list
awscommon-workflow-languagecwlgridenginekubernetesmesospipelinepythonslurmwdlworkflowworkflow-description-language
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 whitaker-io/machine | A library for creating data workflows that can be simple or complex, with features like recursion and memoization. | 159 |
 druths/xp | A tool for creating flexible and self-documenting data science pipelines | 56 |
 pwwang/pipen | A Python-based workflow automation framework that enables easy creation of data processing pipelines | 103 |
 pydoit/doit | An automation tool that enables efficient task management and execution of complex workflows | 1,873 |
 calebwin/pipelines | A language and runtime for crafting massively parallel data pipelines | 374 |
 galaxyproject/galaxy | A platform for data-intensive scientific analysis and workflow management | 1,416 |
 pypyr/pypyr | A tool to automate repetitive tasks in software development pipelines | 599 |
 natcap/taskgraph | A Python library for managing and optimizing computational workflows with parallel processing and data reuse. | 21 |
 scipipe/scipipe | A flexible and efficient way to write and run complex workflows using Go programming language | 1,076 |
 danielgerlag/liteflow | A Python library for running workflows with pluggable persistence and concurrency providers. | 61 |
 pipefunc/pipefunc | Automates and simplifies the creation of function pipelines for efficient execution of scientific workflows. | 217 |
 giacbrd/smartpipeline | A framework for designing and executing concurrent data pipelines with a focus on simplicity and efficiency | 23 |
 4dn-dcic/tibanna | A tool for running genomic pipelines on Amazon cloud using portable workflows in CWL/WDL/Snakemake formats | 70 |
 hhio618/golem-ci | A decentralized task pipeline on Golem.network using Python. | 5 |
 ypares/porcupine | A tool that enables data manipulation and analysis pipelines to be flexible, reusable, and reproducible in different environments | 89 |