soopervisor
Pipeline manager
Tools for exporting and managing workflow pipelines across multiple platforms.
☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.
45 stars
6 watching
19 forks
Language: Python
last commit: about 2 years agoLinked from 1 awesome list
airflowargoargo-workflowsawsdata-sciencekubeflowkubeflow-pipelineskubernetesmachine-learningslurmworkflow
kelvins/awesome-mlopsRelated projects:
| Repository | Description | Stars |
|---|---|---|
 quickube/piper | Automates creation of Kubernetes workflows based on Git branch changes | 22 |
| ploomber/soorgeon | A tool to convert monolithic Jupyter notebooks into maintainable workflows with standardized task execution and output storage. | 78 |
 couler-proj/couler | Provides a unified interface for constructing and managing workflows across different workflow engines. | 919 |
 argoproj/argoproj | A collection of tools and projects for automating and managing Kubernetes workflows and deployments | 630 |
 rookout/piper | Automates the creation of Kubernetes workflows from Git branches | 38 |
 argonne-lcf/balsam | A platform for automating high-throughput workflows across heterogeneous resources in HPC environments. | 77 |
 bloomreach/briefly | A Python library for managing complex workflows with job flow control and resource management. | 106 |
 alexanderrichtertd/plex | A comprehensive pipeline management system for VFX, animation, and game production workflows | 249 |
 natcap/taskgraph | A Python library for managing and optimizing computational workflows with parallel processing and data reuse. | 22 |
 hammerlab/ketrew | Tool for managing and executing complex workflows with multiple dependencies and parameters | 77 |
 agnostiqhq/covalent | A Python library for orchestrating machine-learning and high-performance computing workflows in heterogeneous environments | 785 |
 kubeflow-kale/kale | Simplifies the deployment of Kubeflow Pipelines workflows by providing a graphical interface for Data Scientists to define and deploy pipelines directly from JupyterLab. | 632 |
 kenza-ai/sagify | Simplifies machine learning workflows on AWS SageMaker by providing a unified interface to leverage large language models | 436 |
 jhamrick/nbflow | Tool that supports reproducible workflows with Jupyter Notebooks and SCons. | 160 |
 listyque/tactic-handler | A PySide-based client tool for managing pipelines, assets, and workflows in 3D animation software | 94 |