ploomber
Data pipeline manager
A platform for building and deploying data pipelines using Python, with features for caching, automation, and modularization.
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
4k stars
30 watching
237 forks
Language: Python
last commit: over 1 year ago
Linked from 8 awesome lists
data-engineeringdata-sciencejupyterjupyter-notebooksmachine-learningmlopsnotebookspapermillpipelinespycharmvscodeworkflow
ethicalml/awesome-production-machine-learning
runacapital/awesome-oss-alternatives
pditommaso/awesome-pipeline
kelvins/awesome-mlops
markusschanta/awesome-jupyter
akuity/awesome-argo
vihar/awesome-oss-saas
simomay/find-ossRelated projects:
| Repository | Description | Stars |
|---|---|---|
 pachyderm/pachyderm | Automates data transformations with versioning and lineage tracking for scalable data pipelines | 6,191 |
 pipedreamhq/pipedream | An integration platform that enables developers to automate workflows across multiple applications and services using pre-built components and custom code | 9,075 |
 sveinbjornt/platypus | Creates native Mac applications from command line scripts | 2,858 |
 orchest/orchest | Builds data pipelines by allowing direct coding in Python, R, or Julia without frameworks or YAML files. | 4,091 |
 gradio-app/gradio | Enables rapid creation and deployment of web applications for machine learning models and functions using Python | 34,557 |
 skypilot-org/skypilot | A framework for running AI and batch workloads on any infrastructure, offering unified execution, cost savings, and high GPU availability. | 6,905 |
 wandb/wandb | An AI developer platform to track and manage machine learning models from experimentation to production. | 9,270 |
 bentoml/bentoml | An open-source Python framework for building model inference APIs and serving AI models in production environments. | 7,222 |
 lindb/lindb | A high-performance, distributed time series database with horizontal scalability and high availability | 3,010 |
 ml-tooling/opyrator | Automates conversion of machine learning code into production-ready microservices with web API and GUI. | 3,116 |
 pathwaycom/llm-app | Provides pre-built AI application templates to integrate Large Language Models (LLMs) with various data sources for scalable RAG and enterprise search. | 7,426 |
 pulumi/pulumi | Enables infrastructure provisioning and management using standard programming language features | 22,063 |
 windmill-labs/windmill | An open-source developer platform that integrates scripts with UIs and workflows, allowing for automation of infrastructure and business processes. | 11,216 |
 microsoft/flaml | Automates machine learning workflows and optimizes model performance using large language models and efficient algorithms | 3,968 |