hyperqueue
Workflow manager
A tool that automates the execution of complex workflows on HPC clusters by dynamically allocating resources and load-balancing tasks.
Scheduler for sub-node tasks for HPC systems with batch scheduling
292 stars
7 watching
23 forks
Language: Rust
last commit: almost 2 years agoLinked from 1 awesome list
distributed-computinghpcrusttask-graph
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 ivansharamok/dynamicworkflows | A workflow management tool built on top of Sitecore, allowing users to define and manage dynamic workflows. | 2 |
 liveramp/captain | A lightweight Java workflow engine designed to simplify the composition of microservices into workflows. | 35 |
 hniksic/rust-subprocess | A library providing facilities to execute external processes and pipelines with optional redirection of input/output streams. | 426 |
 centurylinklabs/dray | An engine for managing container-based workflows | 383 |
 illumina/pyflow | A tool to manage tasks in the context of a task dependency graph using Python code. | 146 |
 chaps-io/gush | A workflow runner using Redis and ActiveJob to manage parallel operations with dependencies. | 1,049 |
 bloomreach/briefly | A Python library for managing complex workflows with job flow control and resource management. | 106 |
 hentioe/honeycomb | An Elixir library for managing background tasks with concurrency control and persistent results | 18 |
 agnostiqhq/covalent | A Python library for orchestrating machine-learning and high-performance computing workflows in heterogeneous environments | 785 |
 broadinstitute/cromwell | A workflow management system designed for bioinformatics that simplifies the transition between small and large-scale environments. | 1,004 |
 yoshuawuyts/hyperlapse | Manages and executes processes on a host machine using a distributed process manager | 23 |
| it4innovations/hyperloom | A platform for defining and executing scientific pipelines in distributed environments using C++ and Python. | 16 |
 hey-hoho/schedulemastercore | A distributed task management system with a web UI, supporting cross-platform deployment and multiple scheduling features. | 856 |
 pytask-dev/pytask | A workflow management system that facilitates reproducible data analyses | 115 |
 symfony/workflow | Tools for designing and managing the flow of a system | 598 |