HyperLoom
Pipeline orchestrator
A platform for defining and executing scientific pipelines in distributed environments using C++ and Python.
Platform for defining and executing scientific pipelines in distributed environments
16 stars
7 watching
2 forks
Language: C++
last commit: about 2 years ago
Linked from 1 awesome list
distributed-systemshpcpipelinesschedulertaskworkflow
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 moby/datakit | A tool to orchestrate applications using a version-controlled dataflow | 1,082 |
 hhio618/golem-ci | A decentralized task pipeline on Golem.network using Python. | 5 |
 llnl/maestrowf | A tool to orchestrate computational workflows in high-performance computing environments. | 134 |
 pipefunc/pipefunc | Automates and simplifies the creation of function pipelines for efficient execution of scientific workflows. | 215 |
 huawei/containerops | An orchestration platform for automating DevOps workflows by combining tools and services into a single, GUI-based solution | 338 |
 dagster-io/dagster | An orchestration platform for data pipelines and assets, providing a declarative programming model and integrated lineage and observability. | 11,699 |
 couler-proj/couler | Provides a unified interface for constructing and managing workflows across different workflow engines. | 915 |
 knowsuchagency/orkestra | A Python-based workflow orchestration system built on top of AWS tools and services. | 50 |
 nazar256/parapipe | A non-blocking buffered pipeline library that allows concurrent processing of data while maintaining output order without locks or mutexes. | 31 |
 mafintosh/hyperpipe | Distributes input/output operations between multiple systems using a key-based architecture. | 119 |
 natcap/taskgraph | A Python library for managing and optimizing computational workflows with parallel processing and data reuse. | 21 |
 hyperledger-cacti/cacti | Enables interoperability and integration among multiple blockchain networks by providing a framework for orchestrating transactions across their boundaries. | 344 |
 robrich/orchestrator | A module for sequencing and executing tasks with dependencies in maximum concurrency | 1,225 |
 danielgerlag/conductor | A distributed workflow management system that coordinates services and scripts into complex workflows. | 532 |
 johnsonc/lambdo | A workflow engine for unifying feature engineering and machine learning operations in data analysis pipelines | 1 |