rubra
Pipeline manager
A bioinformatics pipeline system that supports running workflow stages on a distributed compute cluster.
Infrastructure code to support DNA pipeline
38 stars
9 watching
18 forks
Language: Python
last commit: over 11 years agoLinked from 1 awesome list
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 natcap/taskgraph | A Python library for managing and optimizing computational workflows with parallel processing and data reuse. | 22 |
 ssadedin/bpipe | A tool for running and managing bioinformatics pipelines by abstracting away low-level details and providing features such as dependency tracking, transactional management, and parallelism. | 233 |
 samapriya/planet-gee-pipeline-cli | A command-line tool for automating data processing and uploads from Planet's API to Google Earth Engine. | 42 |
 galaxyproject/galaxy | A platform for data-intensive scientific analysis and workflow management | 1,431 |
 montilab/pipeliner | A framework for defining and automating bioinformatics pipelines using Nextflow. | 44 |
 prodmodel/prodmodel | A tool for managing data science pipelines by automating build, testing, and deployment processes while ensuring correctness and performance. | 58 |
 seldonio/tempo | An MLOps Python library that enables data scientists to deploy and orchestrate machine learning pipelines for production-ready inference. | 117 |
 linkedin/brooklin | A distributed system for streaming data between heterogeneous systems with high reliability and throughput at scale | 931 |
 alexanderrichtertd/plex | A comprehensive pipeline management system for VFX, animation, and game production workflows | 249 |
 kakaobrain/torchgpipe | A PyTorch-based library for efficient training of large neural networks using pipeline parallelism and automatic recomputation of gradients. | 820 |
 databiosphere/toil | A workflow management system designed to efficiently run pipelines in various environments. | 901 |
 druths/xp | A tool for creating flexible and self-documenting data science pipelines | 56 |
 kevin-hanselman/dud | A lightweight tool for managing and versioning large data alongside source code in data pipelines | 184 |
 pubs/pubs | A command line tool to organize and manage scientific papers' bibliographic data. | 271 |
 hyfather/pipeline | A package implementing pipelines using goroutines to manage concurrency in Go applications. | 56 |