stolos

Pipeline manager

A task dependency scheduler designed to simplify complex distributed pipelines by managing application execution and job dependencies in a decentralized manner.

A Directed Acyclic Graph task dependency scheduler designed to simplify complex distributed pipelines

GitHub

130 stars
26 watching
17 forks
Language: Python
last commit: over 6 years ago
Linked from 1 awesome list


Backlinks from these awesome lists:

Related projects:

Repository Description Stars
natcap/taskgraph A Python library for managing and optimizing computational workflows with parallel processing and data reuse. 21
sjl/t A simple, flexible command-line todo list manager that helps users focus on completing tasks. 760
hhio618/golem-ci A decentralized task pipeline on Golem.network using Python. 5
illumina/pyflow A tool to manage tasks in the context of a task dependency graph using Python code. 146
hyfather/pipeline A package implementing pipelines using goroutines to manage concurrency in Go applications. 57
bjpop/rubra A bioinformatics pipeline system that supports running workflow stages on a distributed compute cluster. 38
seldonio/tempo An MLOps Python library that enables data scientists to deploy and orchestrate machine learning pipelines for production-ready inference. 116
nlguillemot/pipelineset A utility for managing and reloading graphics pipeline states in Direct3D 12 15
montilab/pipeliner A framework for defining and automating bioinformatics pipelines using Nextflow. 44
samapriya/planet-gee-pipeline-cli A command-line tool for automating data processing and uploads from Planet's API to Google Earth Engine. 42
lightforever/mlcomp A distributed framework for building and managing complex machine learning pipelines with a user-friendly interface. 188
marghidanu/werk A task runner that automates and executes complex workflows with support for parallel jobs, Docker, and real-time output. 27
tbrand/neph A tool for automating repetitive tasks and managing dependencies in command-line workflows. 204
ssadedin/bpipe A tool for running and managing bioinformatics pipelines by abstracting away low-level details and providing features such as dependency tracking, transactional management, and parallelism. 230
ptsochantaris/trailer A tool for managing GitHub pull requests and issues from the command line, with a focus on productivity and repository watchlists. 1,170