flyte
Pipeline Orchestrator
An orchestrator platform that enables the building and deployment of production-grade data pipelines.
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
6k stars
259 watching
659 forks
Language: Go
last commit: 3 days ago
Linked from 4 awesome lists
datadata-analysisdata-sciencedataopsdeclarativefine-tuningflytegolanggrpchacktoberfestkuberneteskubernetes-operatorllmmachine-learningmlopsorchestration-engineproductionpythonscaleworkflow
ethicalml/awesome-production-machine-learning
meirwah/awesome-workflow-engines
pditommaso/awesome-pipeline
vihar/awesome-oss-saasRelated projects:
| Repository | Description | Stars |
|---|---|---|
 airbytehq/airbyte | A platform for building data integration pipelines between various data sources and destinations | 16,184 |
 kestra-io/kestra | An orchestration platform for automating workflows and data processes | 12,971 |
 ml-tooling/opyrator | Automates conversion of machine learning code into production-ready microservices with web API and GUI. | 3,102 |
 netflix/metaflow | A platform that enables scientists and engineers to build, deploy, and manage complex data science projects efficiently | 8,246 |
 apache/dolphinscheduler | A modern data orchestration platform with a low-code interface, supporting high performance and cloud-native workflows. | 12,871 |
 flycheck/flycheck | An Emacs extension that provides on-the-fly syntax checking capabilities. | 2,420 |
 windmill-labs/windmill | An open-source developer platform that integrates scripts with UIs and workflows, allowing for automation of infrastructure and business processes. | 10,864 |
 kubeflow/pipelines | A tool for building and managing machine learning workflows on Kubernetes. | 3,614 |
 couler-proj/couler | Provides a unified interface for constructing and managing workflows across different workflow engines. | 915 |
 skypilot-org/skypilot | A framework for running AI and batch workloads on any infrastructure, offering unified execution, cost savings, and high GPU availability. | 6,801 |
| apache/airflow | A platform to programmatically author, schedule and monitor complex workflows | 37,120 |
 okteto/okteto | Accelerates development of applications in Kubernetes clusters by providing a seamless IDE and tool integration with instant updates | 3,272 |
 ricklamers/gridstudio | A web-based data science application with integration of open source frameworks and languages for data manipulation and visualization. | 8,880 |
 orchest/orchest | Builds data pipelines by allowing direct coding in Python, R, or Julia without frameworks or YAML files. | 4,079 |
 pathwaycom/pathway | An ETL framework that enables real-time data processing and analytics using Python, with support for streaming data, batch processing, machine learning, and integration with various external data sources. | 4,324 |