awesome-mlops

MLOps toolbox

A curated collection of tools and resources supporting the development and deployment of machine learning models and workflows

sunglasses A curated list of awesome MLOps tools

GitHub
4k stars
89 watching
575 forks
Language: Python
last commit: 11 months ago
Linked from 3 awesome lists

aiawesomedata-sciencemachine-learningmachine-learning-engineeringmlmlemlops

Awesome MLOps / AutoML
AutoGluon 8,167 11 months ago Automated machine learning for image, text, tabular, time-series, and multi-modal data
AutoKeras 9,172 11 months ago AutoKeras goal is to make machine learning accessible for everyone
AutoPyTorch 2,385 over 1 year ago Automatic architecture search and hyperparameter optimization for PyTorch
AutoSKLearn 7,667 11 months ago Automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator
EvalML 788 11 months ago A library that builds, optimizes, and evaluates ML pipelines using domain-specific functions
FLAML 3,968 11 months ago Finds accurate ML models automatically, efficiently and economically
H2O AutoML Automates ML workflow, which includes automatic training and tuning of models
MindsDB 26,915 11 months ago AI layer for databases that allows you to effortlessly develop, train and deploy ML models
MLBox 1,500 over 2 years ago MLBox is a powerful Automated Machine Learning python library
Model Search 3,268 over 1 year ago Framework that implements AutoML algorithms for model architecture search at scale
NNI 14,076 over 1 year ago An open source AutoML toolkit for automate machine learning lifecycle

Awesome MLOps / CI/CD for Machine Learning
ClearML 5,740 11 months ago Auto-Magical CI/CD to streamline your ML workflow
CML 4,046 11 months ago Open-source library for implementing CI/CD in machine learning projects
KitOps 577 11 months ago – Open source MLOps project that eases model handoffs between data scientist and DevOps

Awesome MLOps / Cron Job Monitoring
Cronitor Monitor any cron job or scheduled task
HealthchecksIO Simple and effective cron job monitoring

Awesome MLOps / Data Catalog
Amundsen Data discovery and metadata engine for improving the productivity when interacting with data
Apache Atlas Provides open metadata management and governance capabilities to build a data catalog
CKAN 4,509 11 months ago Open-source DMS (data management system) for powering data hubs and data portals
DataHub 10,046 11 months ago LinkedIn's generalized metadata search & discovery tool
Magda 518 11 months ago A federated, open-source data catalog for all your big data and small data
Metacat 1,616 11 months ago Unified metadata exploration API service for Hive, RDS, Teradata, Redshift, S3 and Cassandra
OpenMetadata A Single place to discover, collaborate and get your data right

Awesome MLOps / Data Enrichment
Snorkel 5,826 over 1 year ago A system for quickly generating training data with weak supervision
Upgini 321 11 months ago Enriches training datasets with features from public and community shared data sources

Awesome MLOps / Data Exploration
Apache Zeppelin Enables data-driven, interactive data analytics and collaborative documents
BambooLib 940 over 1 year ago An intuitive GUI for Pandas DataFrames
DataPrep 2,088 over 1 year ago Collect, clean and visualize your data in Python
Google Colab Hosted Jupyter notebook service that requires no setup to use
Jupyter Notebook Web-based notebook environment for interactive computing
JupyterLab The next-generation user interface for Project Jupyter
Jupytext 6,673 11 months ago Jupyter Notebooks as Markdown Documents, Julia, Python or R scripts
Pandas Profiling 12,602 11 months ago Create HTML profiling reports from pandas DataFrame objects
Polynote The polyglot notebook with first-class Scala support

Awesome MLOps / Data Management
Arrikto Dead simple, ultra fast storage for the hybrid Kubernetes world
BlazingSQL 1,938 about 3 years ago A lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF
Delta Lake 7,677 11 months ago Storage layer that brings scalable, ACID transactions to Apache Spark and other engines
Dolt 18,052 11 months ago SQL database that you can fork, clone, branch, merge, push and pull just like a git repository
Dud 184 12 months ago A lightweight CLI tool for versioning data alongside source code and building data pipelines
DVC Management and versioning of datasets and machine learning models
Git LFS An open source Git extension for versioning large files
Hub 8,237 11 months ago A dataset format for creating, storing, and collaborating on AI datasets of any size
Intake 1,015 12 months ago A lightweight set of tools for loading and sharing data in data science projects
lakeFS 4,496 11 months ago Repeatable, atomic and versioned data lake on top of object storage
Marquez 1,800 11 months ago Collect, aggregate, and visualize a data ecosystem's metadata
Milvus 31,283 11 months ago An open source embedding vector similarity search engine powered by Faiss, NMSLIB and Annoy
Pinecone Managed and distributed vector similarity search used with a lightweight SDK
Qdrant 21,001 11 months ago An open source vector similarity search engine with extended filtering support
Quilt 1,328 11 months ago A self-organizing data hub with S3 support

Awesome MLOps / Data Processing
Airflow Platform to programmatically author, schedule, and monitor workflows
Azkaban 4,481 over 1 year ago Batch workflow job scheduler created at LinkedIn to run Hadoop jobs
Dagster 12,055 11 months ago A data orchestrator for machine learning, analytics, and ETL
Hadoop Framework that allows for the distributed processing of large data sets across clusters
OpenRefine 10,968 11 months ago Power tool for working with messy data and improving it
Spark Unified analytics engine for large-scale data processing

Awesome MLOps / Data Validation
Cerberus 3,179 about 1 year ago Lightweight, extensible data validation library for Python
Cleanlab 9,820 11 months ago Python library for data-centric AI and machine learning with messy, real-world data and labels
Great Expectations A Python data validation framework that allows to test your data against datasets
JSON Schema A vocabulary that allows you to annotate and validate JSON documents
TFDV 766 12 months ago An library for exploring and validating machine learning data

Awesome MLOps / Data Visualization
Count SQL/drag-and-drop querying and visualisation tool based on notebooks
Dash 21,641 11 months ago Analytical Web Apps for Python, R, Julia, and Jupyter
Data Studio Reporting solution for power users who want to go beyond the data and dashboards of GA
Facets 7,357 over 2 years ago Visualizations for understanding and analyzing machine learning datasets
Grafana Multi-platform open source analytics and interactive visualization web application
Lux 5,226 over 1 year ago Fast and easy data exploration by automating the visualization and data analysis process
Metabase The simplest, fastest way to get business intelligence and analytics to everyone
Redash Connect to any data source, easily visualize, dashboard and share your data
SolidUI 589 almost 2 years ago AI-generated visualization prototyping and editing platform, support 2D and 3D models
Superset Modern, enterprise-ready business intelligence web application
Tableau Powerful and fastest growing data visualization tool used in the business intelligence industry

Awesome MLOps / Drift Detection
Alibi Detect 2,262 11 months ago An open source Python library focused on outlier, adversarial and drift detection
Frouros 197 11 months ago An open source Python library for drift detection in machine learning systems
TorchDrift 311 about 3 years ago A data and concept drift library for PyTorch

Awesome MLOps / Feature Engineering
Feature Engine 1,956 12 months ago Feature engineering package with SKlearn like functionality
Featuretools 7,304 11 months ago Python library for automated feature engineering
TSFresh 8,486 12 months ago Python library for automatic extraction of relevant features from time series

Awesome MLOps / Feature Store
Butterfree 288 11 months ago A tool for building feature stores. Transform your raw data into beautiful features
ByteHub 58 over 4 years ago An easy-to-use feature store. Optimized for time-series data
Feast End-to-end open source feature store for machine learning
Feathr 1,985 over 1 year ago An enterprise-grade, high performance feature store
Featureform 1,822 11 months ago A Virtual Feature Store. Turn your existing data infrastructure into a feature store
Tecton A fully-managed feature platform built to orchestrate the complete lifecycle of features

Awesome MLOps / Hyperparameter Tuning
Advisor 1,550 almost 6 years ago Open-source implementation of Google Vizier for hyper parameters tuning
Hyperas 2,179 almost 3 years ago A very simple wrapper for convenient hyperparameter optimization
Hyperopt 7,295 about 1 year ago Distributed Asynchronous Hyperparameter Optimization in Python
Katib 1,521 11 months ago Kubernetes-based system for hyperparameter tuning and neural architecture search
KerasTuner 2,860 11 months ago Easy-to-use, scalable hyperparameter optimization framework
Optuna Open source hyperparameter optimization framework to automate hyperparameter search
Scikit Optimize 2,748 over 1 year ago Simple and efficient library to minimize expensive and noisy black-box functions
Talos 1,626 over 1 year ago Hyperparameter Optimization for TensorFlow, Keras and PyTorch
Tune Python library for experiment execution and hyperparameter tuning at any scale

Awesome MLOps / Knowledge Sharing
Knowledge Repo 5,489 about 1 year ago Knowledge sharing platform for data scientists and other technical professions
Kyso One place for data insights so your entire team can learn from your data

Awesome MLOps / Machine Learning Platform
aiWARE aiWARE helps MLOps teams evaluate, deploy, integrate, scale & monitor ML models
Algorithmia Securely govern your machine learning operations with a healthy ML lifecycle
Allegro AI Transform ML/DL research into products. Faster
Bodywork Deploys machine learning projects developed in Python, to Kubernetes
CNVRG An end-to-end machine learning platform to build and deploy AI models at scale
DAGsHub A platform built on open source tools for data, model and pipeline management
Dataiku Platform democratizing access to data and enabling enterprises to build their own path to AI
DataRobot AI platform that democratizes data science and automates the end-to-end ML at scale
Domino One place for your data science tools, apps, results, models, and knowledge
Edge Impulse Platform for creating, optimizing, and deploying AI/ML algorithms for edge devices
envd 2,061 about 1 year ago Machine learning development environment for data science and AI/ML engineering teams
FedML Simplifies the workflow of federated learning anywhere at any scale
Gradient Multicloud CI/CD and MLOps platform for machine learning teams
H2O Open source leader in AI with a mission to democratize AI for everyone
Hopsworks Open-source platform for developing and operating machine learning models at scale
Iguazio Data science platform that automates MLOps with end-to-end machine learning pipelines
Katonic Automate your cycle of intelligence with Katonic MLOps Platform
Knime Create and productionize data science using one easy and intuitive environment
Kubeflow Making deployments of ML workflows on Kubernetes simple, portable and scalable
LynxKite A complete graph data science platform for very large graphs and other datasets
ML Workspace 3,446 over 1 year ago All-in-one web-based IDE specialized for machine learning and data science
MLReef 1,440 about 3 years ago Open source MLOps platform that helps you collaborate, reproduce and share your ML work
Modzy Deploy, connect, run, and monitor machine learning (ML) models in the enterprise and at the edge
Neu.ro MLOps platform that integrates open-source and proprietary tools into client-oriented systems
Omnimizer Simplifies and accelerates MLOps by bridging the gap between ML models and edge hardware
Pachyderm Combines data lineage with end-to-end pipelines on Kubernetes, engineered for the enterprise
Polyaxon A platform for reproducible and scalable machine learning and deep learning on kubernetes
Sagemaker Fully managed service that provides the ability to build, train, and deploy ML models quickly
SAS Viya Cloud native AI, analytic and data management platform that supports the analytics life cycle
Sematic An open-source end-to-end pipelining tool to go from laptop prototype to cloud in no time
SigOpt A platform that makes it easy to track runs, visualize training, and scale hyperparameter tuning
TrueFoundry A Cloud-native MLOps Platform over Kubernetes to simplify training and serving of ML Models
Valohai Takes you from POC to production while managing the whole model lifecycle

Awesome MLOps / Model Fairness and Privacy
AIF360 2,483 11 months ago A comprehensive set of fairness metrics for datasets and machine learning models
Fairlearn 1,974 11 months ago A Python package to assess and improve fairness of machine learning models
Opacus 1,730 11 months ago A library that enables training PyTorch models with differential privacy
TensorFlow Privacy 1,947 12 months ago Library for training machine learning models with privacy for training data

Awesome MLOps / Model Interpretability
Alibi 2,421 11 months ago Open-source Python library enabling ML model inspection and interpretation
Captum 4,982 11 months ago Model interpretability and understanding library for PyTorch
ELI5 265 11 months ago Python package which helps to debug machine learning classifiers and explain their predictions
InterpretML 6,324 11 months ago A toolkit to help understand models and enable responsible machine learning
LIME 11,663 over 1 year ago Explaining the predictions of any machine learning classifier
Lucid 4,678 almost 3 years ago Collection of infrastructure and tools for research in neural network interpretability
SAGE 256 11 months ago For calculating global feature importance using Shapley values
SHAP 23,077 11 months ago A game theoretic approach to explain the output of any machine learning model

Awesome MLOps / Model Lifecycle
Aeromancy 11 about 1 year ago A framework for performing reproducible AI and ML for Weights and Biases
Aim 5,261 11 months ago A super-easy way to record, search and compare 1000s of ML training runs
Cascade 22 11 months ago Library of ML-Engineering tools for rapid prototyping and experiment management
Comet 152 11 months ago Track your datasets, code changes, experimentation history, and models
Guild AI Open source experiment tracking, pipeline automation, and hyperparameter tuning
Keepsake 1,649 11 months ago Version control for machine learning with support to Amazon S3 and Google Cloud Storage
Losswise Makes it easy to track the progress of a machine learning project
MLflow Open source platform for the machine learning lifecycle
ModelDB 1,707 over 1 year ago Open source ML model versioning, metadata, and experiment management
Neptune AI The most lightweight experiment management tool that fits any workflow
Sacred 4,266 12 months ago A tool to help you configure, organize, log and reproduce experiments
Weights and Biases 9,270 11 months ago A tool for visualizing and tracking your machine learning experiments

Awesome MLOps / Model Serving
Banana Host your ML inference code on serverless GPUs and integrate it into your app with one line of code
Beam Develop on serverless GPUs, deploy highly performant APIs, and rapidly prototype ML models
BentoML 7,222 11 months ago Open-source platform for high-performance ML model serving
BudgetML 1,341 over 1 year ago Deploy a ML inference service on a budget in less than 10 lines of code
Cog 8,169 11 months ago Open-source tool that lets you package ML models in a standard, production-ready container
Cortex Machine learning model serving infrastructure
Geniusrise Host inference APIs, bulk inference and fine tune text, vision, audio and multi-modal models
Gradio 34,557 11 months ago Create customizable UI components around your models
GraphPipe Machine learning model deployment made simple
Hydrosphere 271 about 1 year ago Platform for deploying your Machine Learning to production
KFServing 3,716 11 months ago Kubernetes custom resource definition for serving ML models on arbitrary frameworks
LocalAI 27,060 11 months ago Drop-in replacement REST API that’s compatible with OpenAI API specifications for inferencing
Merlin 168 11 months ago A platform for deploying and serving machine learning models
MLEM 718 about 2 years ago Version and deploy your ML models following GitOps principles
Opyrator 3,116 12 months ago Turns your ML code into microservices with web API, interactive GUI, and more
PredictionIO 12,541 almost 5 years ago Event collection, deployment of algorithms, evaluation, querying predictive results via APIs
Quix Serverless platform for processing data streams in real-time with machine learning models
Rune 136 over 3 years ago Provides containers to encapsulate and deploy EdgeML pipelines and applications
Seldon Take your ML projects from POC to production with maximum efficiency and minimal risk
Streamlit 36,168 11 months ago Lets you create apps for your ML projects with deceptively simple Python scripts
TensorFlow Serving Flexible, high-performance serving system for ML models, designed for production
TorchServe 4,238 11 months ago A flexible and easy to use tool for serving PyTorch models
Triton Inference Server 8,460 11 months ago Provides an optimized cloud and edge inferencing solution
Vespa 5,881 11 months ago Store, search, organize and make machine-learned inferences over big data at serving time
Wallaroo.AI A platform for deploying, serving, and optimizing ML models in both cloud and edge environments

Awesome MLOps / Model Testing & Validation
Deepchecks 3,650 11 months ago Open-source package for validating ML models & data, with various checks and suites
Starwhale 214 11 months ago An MLOps/LLMOps platform for model building, evaluation, and fine-tuning
Trubrics 136 over 1 year ago Validate machine learning with data science and domain expert feedback

Awesome MLOps / Optimization Tools
Accelerate 8,056 11 months ago A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
Dask Provides advanced parallelism for analytics, enabling performance at scale for the tools you love
DeepSpeed 35,863 11 months ago Deep learning optimization library that makes distributed training easy, efficient, and effective
Fiber Python distributed computing library for modern computer clusters
Horovod 14,305 11 months ago Distributed deep learning training framework for TensorFlow, Keras, PyTorch, and Apache MXNet
Mahout Distributed linear algebra framework and mathematically expressive Scala DSL
MLlib Apache Spark's scalable machine learning library
Modin 9,942 11 months ago Speed up your Pandas workflows by changing a single line of code
Nebullvm 8,375 over 1 year ago Easy-to-use library to boost AI inference
Nos 636 over 1 year ago Open-source module for running AI workloads on Kubernetes in an optimized way
Petastorm 1,805 almost 2 years ago Enables single machine or distributed training and evaluation of deep learning models
Rapids Gives the ability to execute end-to-end data science and analytics pipelines entirely on GPUs
Ray 34,412 11 months ago Fast and simple framework for building and running distributed applications
Singa Apache top level project, focusing on distributed training of DL and ML models
Tpot 9,776 over 1 year ago Automated ML tool that optimizes machine learning pipelines using genetic programming

Awesome MLOps / Simplification Tools
Chassis Turns models into ML-friendly containers that run just about anywhere
Hermione 207 over 2 years ago Help Data Scientists on setting up more organized codes, in a quicker and simpler way
Hydra 8,876 12 months ago A framework for elegantly configuring complex applications
Koalas 3,343 over 1 year ago Pandas API on Apache Spark. Makes data scientists more productive when interacting with big data
Ludwig 11,236 11 months ago Allows users to train and test deep learning models without the need to write code
MLNotify 343 over 3 years ago No need to keep checking your training, just one import line and you'll know the second it's done
PyCaret Open source, low-code machine learning library in Python
Sagify 436 over 1 year ago A CLI utility to train and deploy ML/DL models on AWS SageMaker
Soopervisor 45 about 1 year ago Export ML projects to Kubernetes (Argo workflows), Airflow, AWS Batch, and SLURM
Soorgeon 78 about 1 year ago Convert monolithic Jupyter notebooks into maintainable pipelines
TrainGenerator 1,370 about 2 years ago A web app to generate template code for machine learning
Turi Create 11,202 about 2 years ago Simplifies the development of custom machine learning models

Awesome MLOps / Visual Analysis and Debugging
Aporia Observability with customized monitoring and explainability for ML models
Arize A free end-to-end ML observability and model monitoring platform
Evidently 5,519 11 months ago Interactive reports to analyze ML models during validation or production monitoring
Fiddler Monitor, explain, and analyze your AI in production
Manifold 1,651 over 2 years ago A model-agnostic visual debugging tool for machine learning
NannyML 1,998 about 1 year ago Algorithm capable of fully capturing the impact of data drift on performance
Netron 28,684 11 months ago Visualizer for neural network, deep learning, and machine learning models
Opik 2,588 11 months ago Evaluate, test, and ship LLM applications with a suite of observability tools
Phoenix MLOps in a Notebook for troubleshooting and fine-tuning generative LLM, CV, and tabular models
Radicalbit 73 11 months ago The open source solution for monitoring your AI models in production
Superwise Fully automated, enterprise-grade model observability in a self-service SaaS platform
Whylogs 2,664 11 months ago The open source standard for data logging. Enables ML monitoring and observability
Yellowbrick 4,304 about 1 year ago Visual analysis and diagnostic tools to facilitate machine learning model selection

Awesome MLOps / Workflow Tools
Argo 15,155 11 months ago Open source container-native workflow engine for orchestrating parallel jobs on Kubernetes
Automate Studio Rapidly build & deploy AI-powered workflows
Couler 919 about 1 year ago Unified interface for constructing and managing workflows on different workflow engines
dstack 1,633 11 months ago An open-core tool to automate data and training workflows
Flyte Easy to create concurrent, scalable, and maintainable workflows for machine learning
Hamilton 1,900 11 months ago A scalable general purpose micro-framework for defining dataflows
Kale 632 almost 3 years ago Aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows
Kedro 10,050 11 months ago Library that implements software engineering best-practice for data and ML pipelines
Luigi 17,950 11 months ago Python module that helps you build complex pipelines of batch jobs
Metaflow Human-friendly lib that helps scientists and engineers build and manage data science projects
MLRun 1,458 11 months ago Generic mechanism for data scientists to build, run, and monitor ML tasks and pipelines
Orchest 4,091 over 2 years ago Visual pipeline editor and workflow orchestrator with an easy to use UI and based on Kubernetes
Ploomber 3,530 about 1 year ago Write maintainable, production-ready pipelines. Develop locally, deploy to the cloud
Prefect A workflow management system, designed for modern infrastructure
VDP 2,181 11 months ago An open-source tool to seamlessly integrate AI for unstructured data into the modern data stack
Wordware A web-hosted IDE where non-technical domain experts can build task-specific AI agents
ZenML 4,261 11 months ago An extensible open-source MLOps framework to create reproducible pipelines

Resources / Articles
A Tour of End-to-End Machine Learning Platforms (Databaseline)
Continuous Delivery for Machine Learning (Martin Fowler)
Machine Learning Operations (MLOps): Overview, Definition, and Architecture (arXiv)
MLOps Roadmap: A Complete MLOps Career Guide (Scaler Blogs)
MLOps: Continuous delivery and automation pipelines in machine learning (Google)
MLOps: Machine Learning as an Engineering Discipline (Medium)
Rules of Machine Learning: Best Practices for ML Engineering (Google)
The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction (Google)
What Is MLOps? (NVIDIA)

Resources / Books
Beginning MLOps with MLFlow (Apress)
Building Machine Learning Pipelines (O'Reilly)
Building Machine Learning Powered Applications (O'Reilly)
Deep Learning in Production (AI Summer)
Designing Machine Learning Systems (O'Reilly)
Engineering MLOps (Packt)
Implementing MLOps in the Enterprise (O'Reilly)
Introducing MLOps (O'Reilly)
Kubeflow for Machine Learning (O'Reilly)
Kubeflow Operations Guide (O'Reilly)
Machine Learning Design Patterns (O'Reilly)
Machine Learning Engineering in Action (Manning)
ML Ops: Operationalizing Data Science (O'Reilly)
MLOps Engineering at Scale (Manning)
MLOps Lifecycle Toolkit (Apress)
Practical Deep Learning at Scale with MLflow (Packt)
Practical MLOps (O'Reilly)
Production-Ready Applied Deep Learning (Packt)
Reliable Machine Learning (O'Reilly)
The Machine Learning Solutions Architect Handbook (Packt)

Resources / Events
apply() - The ML data engineering conference
MLOps Conference - Keynotes and Panels
MLOps World: Machine Learning in Production Conference
NormConf - The Normcore Tech Conference
Stanford MLSys Seminar Series

Resources / Other Lists
Applied ML 27,407 over 1 year ago
Awesome AutoML Papers 4,035 over 1 year ago
Awesome AutoML 870 about 1 year ago
Awesome Data Science 25,352 12 months ago
Awesome DataOps 163 about 1 year ago
Awesome Deep Learning 24,435 over 1 year ago
Awesome Game Datasets 763 12 months ago (includes AI content)
Awesome Machine Learning 66,380 11 months ago
Awesome MLOps 12,684 12 months ago
Awesome Production Machine Learning 17,721 11 months ago
Awesome Python 227,763 about 1 year ago
Deep Learning in Production 4,313 12 months ago

Resources / Podcasts
Kubernetes Podcast from Google
Machine Learning – Software Engineering Daily
MLOps.community
Pipeline Conversation
Practical AI: Machine Learning, Data Science
This Week in Machine Learning & AI
True ML Talks

Resources / Slack
Kubeflow Workspace
MLOps Community Wokspace

Resources / Websites
Feature Stores for ML
Made with ML 37,816 about 1 year ago
ML-Ops
MLOps Community
MLOps Guide
MLOps Now

Backlinks from these awesome lists:

More related projects: