flyte
Pipeline Orchestrator
An orchestrator platform that enables the building and deployment of production-grade data pipelines.
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
6k stars
259 watching
667 forks
Language: Go
last commit: 2 months ago
Linked from 4 awesome lists
datadata-analysisdata-sciencedataopsdeclarativefine-tuningflytegolanggrpchacktoberfestkuberneteskubernetes-operatorllmmachine-learningmlopsorchestration-engineproductionpythonscaleworkflow
ethicalml/awesome-production-machine-learning
meirwah/awesome-workflow-engines
pditommaso/awesome-pipeline
vihar/awesome-oss-saasRelated projects:
| Repository | Description | Stars |
|---|---|---|
 airbytehq/airbyte | A platform for building data integration pipelines between various data sources and destinations | 16,441 |
 kestra-io/kestra | An event-driven orchestration platform that automates workflows using a simple configuration and provides a rich plugin ecosystem. | 14,708 |
 ml-tooling/opyrator | Automates conversion of machine learning code into production-ready microservices with web API and GUI. | 3,116 |
 netflix/metaflow | A platform that enables the development, scaling, and deployment of machine learning systems, providing tools for data science projects. | 8,341 |
 apache/dolphinscheduler | A modern data orchestration platform with a low-code interface, supporting high performance and cloud-native workflows. | 13,128 |
 flycheck/flycheck | An Emacs extension that provides on-the-fly syntax checking capabilities. | 2,423 |
 windmill-labs/windmill | An open-source developer platform that integrates scripts with UIs and workflows, allowing for automation of infrastructure and business processes. | 11,216 |
 kubeflow/pipelines | A tool for building and managing machine learning workflows on Kubernetes. | 3,634 |
 couler-proj/couler | Provides a unified interface for constructing and managing workflows across different workflow engines. | 919 |
 skypilot-org/skypilot | A framework for running AI and batch workloads on any infrastructure, offering unified execution, cost savings, and high GPU availability. | 6,905 |
| apache/airflow | A platform to programmatically author, schedule and monitor complex workflows | 37,580 |
 okteto/okteto | Accelerates development of applications in Kubernetes clusters by providing a seamless IDE and tool integration with instant updates | 3,287 |
 ricklamers/gridstudio | A web-based data science application with integration of open source frameworks and languages for data manipulation and visualization. | 8,882 |
 orchest/orchest | Builds data pipelines by allowing direct coding in Python, R, or Julia without frameworks or YAML files. | 4,091 |
 pathwaycom/pathway | A Python-based ETL framework for stream processing and real-time analytics with a scalable Rust engine. | 7,174 |