HyperLoom
Pipeline orchestrator
A platform for defining and executing scientific pipelines in distributed environments using C++ and Python.
Platform for defining and executing scientific pipelines in distributed environments
16 stars
7 watching
2 forks
Language: C++
last commit: almost 4 years agoLinked from 1 awesome list
distributed-systemshpcpipelinesschedulertaskworkflow
pditommaso/awesome-pipelineRelated projects:
| Repository | Description | Stars |
|---|---|---|
 moby/datakit | A tool to orchestrate applications using a version-controlled dataflow | 1,083 |
 hhio618/golem-ci | A decentralized task pipeline on Golem.network using Python. | 5 |
 llnl/maestrowf | A tool to orchestrate computational workflows in high-performance computing environments. | 139 |
 pipefunc/pipefunc | Automates and simplifies the creation of function pipelines for efficient execution of scientific workflows. | 230 |
 huawei/containerops | An orchestration platform for automating DevOps workflows by combining tools and services into a single, GUI-based solution | 339 |
 dagster-io/dagster | An orchestration platform for data pipelines and assets, providing a declarative programming model and integrated lineage and observability. | 12,055 |
 couler-proj/couler | Provides a unified interface for constructing and managing workflows across different workflow engines. | 919 |
 knowsuchagency/orkestra | A Python-based workflow orchestration system built on top of AWS tools and services. | 50 |
 nazar256/parapipe | A library that provides a concurrent, non-blocking buffered pipeline for structuring and scaling applications. | 33 |
 mafintosh/hyperpipe | Distributes input/output operations between multiple systems using a key-based architecture. | 119 |
 natcap/taskgraph | A Python library for managing and optimizing computational workflows with parallel processing and data reuse. | 22 |
 hyperledger-cacti/cacti | Enables interoperability and integration among multiple blockchain networks by providing a framework for orchestrating transactions across their boundaries. | 347 |
 robrich/orchestrator | A module for sequencing and executing tasks with dependencies in maximum concurrency | 1,225 |
 danielgerlag/conductor | A distributed workflow management system that coordinates services and scripts into complex workflows. | 538 |
 johnsonc/lambdo | A workflow engine for unifying feature engineering and machine learning operations in data analysis pipelines | 1 |