taskgraph
Pipeline manager
A Python library for managing and optimizing computational workflows with parallel processing and data reuse.
22 stars
7 watching
8 forks
Language: Python
last commit: over 2 years agoLinked from 1 awesome list
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 bjpop/rubra | A bioinformatics pipeline system that supports running workflow stages on a distributed compute cluster. | 38 |
 sailthru/stolos | A task dependency scheduler designed to simplify complex distributed pipelines by managing application execution and job dependencies in a decentralized manner. | 130 |
 nlguillemot/pipelineset | A utility for managing and reloading graphics pipeline states in Direct3D 12 | 15 |
 samapriya/planet-gee-pipeline-cli | A command-line tool for automating data processing and uploads from Planet's API to Google Earth Engine. | 42 |
 galaxyproject/galaxy | A platform for data-intensive scientific analysis and workflow management | 1,431 |
 illumina/pyflow | A tool to manage tasks in the context of a task dependency graph using Python code. | 146 |
 ssadedin/bpipe | A tool for running and managing bioinformatics pipelines by abstracting away low-level details and providing features such as dependency tracking, transactional management, and parallelism. | 233 |
 seldonio/tempo | An MLOps Python library that enables data scientists to deploy and orchestrate machine learning pipelines for production-ready inference. | 117 |
 databiosphere/toil | A workflow management system designed to efficiently run pipelines in various environments. | 901 |
 montilab/pipeliner | A framework for defining and automating bioinformatics pipelines using Nextflow. | 44 |
 hhio618/golem-ci | A decentralized task pipeline on Golem.network using Python. | 5 |
 pwwang/pipen | A Python-based workflow automation framework that enables easy creation of data processing pipelines | 105 |
 gc3pie/gc3pie | A suite of Python classes and tools for managing computational workflows on diverse Grids and clusters | 44 |
 prodmodel/prodmodel | A tool for managing data science pipelines by automating build, testing, and deployment processes while ensuring correctness and performance. | 58 |
 fluidattacks/makes | A framework for building and managing CI/CD pipelines and application environments with cryptographic signed dependencies. | 461 |