awesome-mlops

MLOps toolbox

A curated collection of tools and resources supporting the development and deployment of machine learning models and workflows

sunglasses A curated list of awesome MLOps tools

GitHub

4k stars
89 watching
575 forks
Language: Python
last commit: almost 2 years ago
Linked from 3 awesome lists

aiawesomedata-sciencemachine-learningmachine-learning-engineeringmlmlemlops

Awesome MLOps / AutoML

AutoGluon8,167almost 2 years agoAutomated machine learning for image, text, tabular, time-series, and multi-modal data
AutoKeras9,172almost 2 years agoAutoKeras goal is to make machine learning accessible for everyone
AutoPyTorch2,385over 2 years agoAutomatic architecture search and hyperparameter optimization for PyTorch
AutoSKLearn7,667almost 2 years agoAutomated machine learning toolkit and a drop-in replacement for a scikit-learn estimator
EvalML788almost 2 years agoA library that builds, optimizes, and evaluates ML pipelines using domain-specific functions
FLAML3,968almost 2 years agoFinds accurate ML models automatically, efficiently and economically
H2O AutoMLAutomates ML workflow, which includes automatic training and tuning of models
MindsDB26,915almost 2 years agoAI layer for databases that allows you to effortlessly develop, train and deploy ML models
MLBox1,500about 3 years agoMLBox is a powerful Automated Machine Learning python library
Model Search3,268about 2 years agoFramework that implements AutoML algorithms for model architecture search at scale
NNI14,076about 2 years agoAn open source AutoML toolkit for automate machine learning lifecycle

Awesome MLOps / CI/CD for Machine Learning

ClearML5,740almost 2 years agoAuto-Magical CI/CD to streamline your ML workflow
CML4,046almost 2 years agoOpen-source library for implementing CI/CD in machine learning projects
KitOps577almost 2 years ago– Open source MLOps project that eases model handoffs between data scientist and DevOps

Awesome MLOps / Cron Job Monitoring

CronitorMonitor any cron job or scheduled task
HealthchecksIOSimple and effective cron job monitoring

Awesome MLOps / Data Catalog

AmundsenData discovery and metadata engine for improving the productivity when interacting with data
Apache AtlasProvides open metadata management and governance capabilities to build a data catalog
CKAN4,509almost 2 years agoOpen-source DMS (data management system) for powering data hubs and data portals
DataHub10,046almost 2 years agoLinkedIn's generalized metadata search & discovery tool
Magda518almost 2 years agoA federated, open-source data catalog for all your big data and small data
Metacat1,616almost 2 years agoUnified metadata exploration API service for Hive, RDS, Teradata, Redshift, S3 and Cassandra
OpenMetadataA Single place to discover, collaborate and get your data right

Awesome MLOps / Data Enrichment

Snorkel5,826over 2 years agoA system for quickly generating training data with weak supervision
Upgini321almost 2 years agoEnriches training datasets with features from public and community shared data sources

Awesome MLOps / Data Exploration

Apache ZeppelinEnables data-driven, interactive data analytics and collaborative documents
BambooLib940over 2 years agoAn intuitive GUI for Pandas DataFrames
DataPrep2,088over 2 years agoCollect, clean and visualize your data in Python
Google ColabHosted Jupyter notebook service that requires no setup to use
Jupyter NotebookWeb-based notebook environment for interactive computing
JupyterLabThe next-generation user interface for Project Jupyter
Jupytext6,673almost 2 years agoJupyter Notebooks as Markdown Documents, Julia, Python or R scripts
Pandas Profiling12,602almost 2 years agoCreate HTML profiling reports from pandas DataFrame objects
PolynoteThe polyglot notebook with first-class Scala support

Awesome MLOps / Data Management

ArriktoDead simple, ultra fast storage for the hybrid Kubernetes world
BlazingSQL1,938about 4 years agoA lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF
Delta Lake7,677almost 2 years agoStorage layer that brings scalable, ACID transactions to Apache Spark and other engines
Dolt18,052almost 2 years agoSQL database that you can fork, clone, branch, merge, push and pull just like a git repository
Dud184almost 2 years agoA lightweight CLI tool for versioning data alongside source code and building data pipelines
DVCManagement and versioning of datasets and machine learning models
Git LFSAn open source Git extension for versioning large files
Hub8,237almost 2 years agoA dataset format for creating, storing, and collaborating on AI datasets of any size
Intake1,015almost 2 years agoA lightweight set of tools for loading and sharing data in data science projects
lakeFS4,496almost 2 years agoRepeatable, atomic and versioned data lake on top of object storage
Marquez1,800almost 2 years agoCollect, aggregate, and visualize a data ecosystem's metadata
Milvus31,283almost 2 years agoAn open source embedding vector similarity search engine powered by Faiss, NMSLIB and Annoy
PineconeManaged and distributed vector similarity search used with a lightweight SDK
Qdrant21,001almost 2 years agoAn open source vector similarity search engine with extended filtering support
Quilt1,328almost 2 years agoA self-organizing data hub with S3 support

Awesome MLOps / Data Processing

AirflowPlatform to programmatically author, schedule, and monitor workflows
Azkaban4,481about 2 years agoBatch workflow job scheduler created at LinkedIn to run Hadoop jobs
Dagster12,055almost 2 years agoA data orchestrator for machine learning, analytics, and ETL
HadoopFramework that allows for the distributed processing of large data sets across clusters
OpenRefine10,968almost 2 years agoPower tool for working with messy data and improving it
SparkUnified analytics engine for large-scale data processing

Awesome MLOps / Data Validation

Cerberus3,179about 2 years agoLightweight, extensible data validation library for Python
Cleanlab9,820almost 2 years agoPython library for data-centric AI and machine learning with messy, real-world data and labels
Great ExpectationsA Python data validation framework that allows to test your data against datasets
JSON SchemaA vocabulary that allows you to annotate and validate JSON documents
TFDV766almost 2 years agoAn library for exploring and validating machine learning data

Awesome MLOps / Data Visualization

CountSQL/drag-and-drop querying and visualisation tool based on notebooks
Dash21,641almost 2 years agoAnalytical Web Apps for Python, R, Julia, and Jupyter
Data StudioReporting solution for power users who want to go beyond the data and dashboards of GA
Facets7,357over 3 years agoVisualizations for understanding and analyzing machine learning datasets
GrafanaMulti-platform open source analytics and interactive visualization web application
Lux5,226over 2 years agoFast and easy data exploration by automating the visualization and data analysis process
MetabaseThe simplest, fastest way to get business intelligence and analytics to everyone
RedashConnect to any data source, easily visualize, dashboard and share your data
SolidUI589over 2 years agoAI-generated visualization prototyping and editing platform, support 2D and 3D models
SupersetModern, enterprise-ready business intelligence web application
TableauPowerful and fastest growing data visualization tool used in the business intelligence industry

Awesome MLOps / Drift Detection

Alibi Detect2,262almost 2 years agoAn open source Python library focused on outlier, adversarial and drift detection
Frouros197almost 2 years agoAn open source Python library for drift detection in machine learning systems
TorchDrift311about 4 years agoA data and concept drift library for PyTorch

Awesome MLOps / Feature Engineering

Feature Engine1,956almost 2 years agoFeature engineering package with SKlearn like functionality
Featuretools7,304almost 2 years agoPython library for automated feature engineering
TSFresh8,486almost 2 years agoPython library for automatic extraction of relevant features from time series

Awesome MLOps / Feature Store

Butterfree288almost 2 years agoA tool for building feature stores. Transform your raw data into beautiful features
ByteHub58over 5 years agoAn easy-to-use feature store. Optimized for time-series data
FeastEnd-to-end open source feature store for machine learning
Feathr1,985over 2 years agoAn enterprise-grade, high performance feature store
Featureform1,822almost 2 years agoA Virtual Feature Store. Turn your existing data infrastructure into a feature store
TectonA fully-managed feature platform built to orchestrate the complete lifecycle of features

Awesome MLOps / Hyperparameter Tuning

Advisor1,550almost 7 years agoOpen-source implementation of Google Vizier for hyper parameters tuning
Hyperas2,179over 3 years agoA very simple wrapper for convenient hyperparameter optimization
Hyperopt7,295almost 2 years agoDistributed Asynchronous Hyperparameter Optimization in Python
Katib1,521almost 2 years agoKubernetes-based system for hyperparameter tuning and neural architecture search
KerasTuner2,860almost 2 years agoEasy-to-use, scalable hyperparameter optimization framework
OptunaOpen source hyperparameter optimization framework to automate hyperparameter search
Scikit Optimize2,748over 2 years agoSimple and efficient library to minimize expensive and noisy black-box functions
Talos1,626over 2 years agoHyperparameter Optimization for TensorFlow, Keras and PyTorch
TunePython library for experiment execution and hyperparameter tuning at any scale

Awesome MLOps / Knowledge Sharing

Knowledge Repo5,489about 2 years agoKnowledge sharing platform for data scientists and other technical professions
KysoOne place for data insights so your entire team can learn from your data

Awesome MLOps / Machine Learning Platform

aiWAREaiWARE helps MLOps teams evaluate, deploy, integrate, scale & monitor ML models
AlgorithmiaSecurely govern your machine learning operations with a healthy ML lifecycle
Allegro AITransform ML/DL research into products. Faster
BodyworkDeploys machine learning projects developed in Python, to Kubernetes
CNVRGAn end-to-end machine learning platform to build and deploy AI models at scale
DAGsHubA platform built on open source tools for data, model and pipeline management
DataikuPlatform democratizing access to data and enabling enterprises to build their own path to AI
DataRobotAI platform that democratizes data science and automates the end-to-end ML at scale
DominoOne place for your data science tools, apps, results, models, and knowledge
Edge ImpulsePlatform for creating, optimizing, and deploying AI/ML algorithms for edge devices
envd2,061almost 2 years agoMachine learning development environment for data science and AI/ML engineering teams
FedMLSimplifies the workflow of federated learning anywhere at any scale
GradientMulticloud CI/CD and MLOps platform for machine learning teams
H2OOpen source leader in AI with a mission to democratize AI for everyone
HopsworksOpen-source platform for developing and operating machine learning models at scale
IguazioData science platform that automates MLOps with end-to-end machine learning pipelines
KatonicAutomate your cycle of intelligence with Katonic MLOps Platform
KnimeCreate and productionize data science using one easy and intuitive environment
KubeflowMaking deployments of ML workflows on Kubernetes simple, portable and scalable
LynxKiteA complete graph data science platform for very large graphs and other datasets
ML Workspace3,446about 2 years agoAll-in-one web-based IDE specialized for machine learning and data science
MLReef1,440almost 4 years agoOpen source MLOps platform that helps you collaborate, reproduce and share your ML work
ModzyDeploy, connect, run, and monitor machine learning (ML) models in the enterprise and at the edge
Neu.roMLOps platform that integrates open-source and proprietary tools into client-oriented systems
OmnimizerSimplifies and accelerates MLOps by bridging the gap between ML models and edge hardware
PachydermCombines data lineage with end-to-end pipelines on Kubernetes, engineered for the enterprise
PolyaxonA platform for reproducible and scalable machine learning and deep learning on kubernetes
SagemakerFully managed service that provides the ability to build, train, and deploy ML models quickly
SAS ViyaCloud native AI, analytic and data management platform that supports the analytics life cycle
SematicAn open-source end-to-end pipelining tool to go from laptop prototype to cloud in no time
SigOptA platform that makes it easy to track runs, visualize training, and scale hyperparameter tuning
TrueFoundryA Cloud-native MLOps Platform over Kubernetes to simplify training and serving of ML Models
ValohaiTakes you from POC to production while managing the whole model lifecycle

Awesome MLOps / Model Fairness and Privacy

AIF3602,483almost 2 years agoA comprehensive set of fairness metrics for datasets and machine learning models
Fairlearn1,974almost 2 years agoA Python package to assess and improve fairness of machine learning models
Opacus1,730almost 2 years agoA library that enables training PyTorch models with differential privacy
TensorFlow Privacy1,947almost 2 years agoLibrary for training machine learning models with privacy for training data

Awesome MLOps / Model Interpretability

Alibi2,421almost 2 years agoOpen-source Python library enabling ML model inspection and interpretation
Captum4,982almost 2 years agoModel interpretability and understanding library for PyTorch
ELI5265almost 2 years agoPython package which helps to debug machine learning classifiers and explain their predictions
InterpretML6,324almost 2 years agoA toolkit to help understand models and enable responsible machine learning
LIME11,663about 2 years agoExplaining the predictions of any machine learning classifier
Lucid4,678over 3 years agoCollection of infrastructure and tools for research in neural network interpretability
SAGE256almost 2 years agoFor calculating global feature importance using Shapley values
SHAP23,077almost 2 years agoA game theoretic approach to explain the output of any machine learning model

Awesome MLOps / Model Lifecycle

Aeromancy11almost 2 years agoA framework for performing reproducible AI and ML for Weights and Biases
Aim5,261almost 2 years agoA super-easy way to record, search and compare 1000s of ML training runs
Cascade22almost 2 years agoLibrary of ML-Engineering tools for rapid prototyping and experiment management
Comet152almost 2 years agoTrack your datasets, code changes, experimentation history, and models
Guild AIOpen source experiment tracking, pipeline automation, and hyperparameter tuning
Keepsake1,649almost 2 years agoVersion control for machine learning with support to Amazon S3 and Google Cloud Storage
LosswiseMakes it easy to track the progress of a machine learning project
MLflowOpen source platform for the machine learning lifecycle
ModelDB1,707about 2 years agoOpen source ML model versioning, metadata, and experiment management
Neptune AIThe most lightweight experiment management tool that fits any workflow
Sacred4,266almost 2 years agoA tool to help you configure, organize, log and reproduce experiments
Weights and Biases9,270almost 2 years agoA tool for visualizing and tracking your machine learning experiments

Awesome MLOps / Model Serving

BananaHost your ML inference code on serverless GPUs and integrate it into your app with one line of code
BeamDevelop on serverless GPUs, deploy highly performant APIs, and rapidly prototype ML models
BentoML7,222almost 2 years agoOpen-source platform for high-performance ML model serving
BudgetML1,341over 2 years agoDeploy a ML inference service on a budget in less than 10 lines of code
Cog8,169almost 2 years agoOpen-source tool that lets you package ML models in a standard, production-ready container
CortexMachine learning model serving infrastructure
GeniusriseHost inference APIs, bulk inference and fine tune text, vision, audio and multi-modal models
Gradio34,557almost 2 years agoCreate customizable UI components around your models
GraphPipeMachine learning model deployment made simple
Hydrosphere271almost 2 years agoPlatform for deploying your Machine Learning to production
KFServing3,716almost 2 years agoKubernetes custom resource definition for serving ML models on arbitrary frameworks
LocalAI27,060almost 2 years agoDrop-in replacement REST API that’s compatible with OpenAI API specifications for inferencing
Merlin168almost 2 years agoA platform for deploying and serving machine learning models
MLEM718about 3 years agoVersion and deploy your ML models following GitOps principles
Opyrator3,116almost 2 years agoTurns your ML code into microservices with web API, interactive GUI, and more
PredictionIO12,541over 5 years agoEvent collection, deployment of algorithms, evaluation, querying predictive results via APIs
QuixServerless platform for processing data streams in real-time with machine learning models
Rune136about 4 years agoProvides containers to encapsulate and deploy EdgeML pipelines and applications
SeldonTake your ML projects from POC to production with maximum efficiency and minimal risk
Streamlit36,168almost 2 years agoLets you create apps for your ML projects with deceptively simple Python scripts
TensorFlow ServingFlexible, high-performance serving system for ML models, designed for production
TorchServe4,238almost 2 years agoA flexible and easy to use tool for serving PyTorch models
Triton Inference Server8,460almost 2 years agoProvides an optimized cloud and edge inferencing solution
Vespa5,881almost 2 years agoStore, search, organize and make machine-learned inferences over big data at serving time
Wallaroo.AIA platform for deploying, serving, and optimizing ML models in both cloud and edge environments

Awesome MLOps / Model Testing & Validation

Deepchecks3,650almost 2 years agoOpen-source package for validating ML models & data, with various checks and suites
Starwhale214almost 2 years agoAn MLOps/LLMOps platform for model building, evaluation, and fine-tuning
Trubrics136over 2 years agoValidate machine learning with data science and domain expert feedback

Awesome MLOps / Optimization Tools

Accelerate8,056almost 2 years agoA simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
DaskProvides advanced parallelism for analytics, enabling performance at scale for the tools you love
DeepSpeed35,863almost 2 years agoDeep learning optimization library that makes distributed training easy, efficient, and effective
FiberPython distributed computing library for modern computer clusters
Horovod14,305almost 2 years agoDistributed deep learning training framework for TensorFlow, Keras, PyTorch, and Apache MXNet
MahoutDistributed linear algebra framework and mathematically expressive Scala DSL
MLlibApache Spark's scalable machine learning library
Modin9,942almost 2 years agoSpeed up your Pandas workflows by changing a single line of code
Nebullvm8,375about 2 years agoEasy-to-use library to boost AI inference
Nos636over 2 years agoOpen-source module for running AI workloads on Kubernetes in an optimized way
Petastorm1,805almost 3 years agoEnables single machine or distributed training and evaluation of deep learning models
RapidsGives the ability to execute end-to-end data science and analytics pipelines entirely on GPUs
Ray34,412almost 2 years agoFast and simple framework for building and running distributed applications
SingaApache top level project, focusing on distributed training of DL and ML models
Tpot9,776about 2 years agoAutomated ML tool that optimizes machine learning pipelines using genetic programming

Awesome MLOps / Simplification Tools

ChassisTurns models into ML-friendly containers that run just about anywhere
Hermione207over 3 years agoHelp Data Scientists on setting up more organized codes, in a quicker and simpler way
Hydra8,876almost 2 years agoA framework for elegantly configuring complex applications
Koalas3,343over 2 years agoPandas API on Apache Spark. Makes data scientists more productive when interacting with big data
Ludwig11,236almost 2 years agoAllows users to train and test deep learning models without the need to write code
MLNotify343over 4 years agoNo need to keep checking your training, just one import line and you'll know the second it's done
PyCaretOpen source, low-code machine learning library in Python
Sagify436over 2 years agoA CLI utility to train and deploy ML/DL models on AWS SageMaker
Soopervisor45about 2 years agoExport ML projects to Kubernetes (Argo workflows), Airflow, AWS Batch, and SLURM
Soorgeon78about 2 years agoConvert monolithic Jupyter notebooks into maintainable pipelines
TrainGenerator1,370about 3 years agoA web app to generate template code for machine learning
Turi Create11,202almost 3 years agoSimplifies the development of custom machine learning models

Awesome MLOps / Visual Analysis and Debugging

AporiaObservability with customized monitoring and explainability for ML models
ArizeA free end-to-end ML observability and model monitoring platform
Evidently5,519almost 2 years agoInteractive reports to analyze ML models during validation or production monitoring
FiddlerMonitor, explain, and analyze your AI in production
Manifold1,651over 3 years agoA model-agnostic visual debugging tool for machine learning
NannyML1,998almost 2 years agoAlgorithm capable of fully capturing the impact of data drift on performance
Netron28,684almost 2 years agoVisualizer for neural network, deep learning, and machine learning models
Opik2,588almost 2 years agoEvaluate, test, and ship LLM applications with a suite of observability tools
PhoenixMLOps in a Notebook for troubleshooting and fine-tuning generative LLM, CV, and tabular models
Radicalbit73almost 2 years agoThe open source solution for monitoring your AI models in production
SuperwiseFully automated, enterprise-grade model observability in a self-service SaaS platform
Whylogs2,664almost 2 years agoThe open source standard for data logging. Enables ML monitoring and observability
Yellowbrick4,304about 2 years agoVisual analysis and diagnostic tools to facilitate machine learning model selection

Awesome MLOps / Workflow Tools

Argo15,155almost 2 years agoOpen source container-native workflow engine for orchestrating parallel jobs on Kubernetes
Automate StudioRapidly build & deploy AI-powered workflows
Couler919almost 2 years agoUnified interface for constructing and managing workflows on different workflow engines
dstack1,633almost 2 years agoAn open-core tool to automate data and training workflows
FlyteEasy to create concurrent, scalable, and maintainable workflows for machine learning
Hamilton1,900almost 2 years agoA scalable general purpose micro-framework for defining dataflows
Kale632over 3 years agoAims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows
Kedro10,050almost 2 years agoLibrary that implements software engineering best-practice for data and ML pipelines
Luigi17,950almost 2 years agoPython module that helps you build complex pipelines of batch jobs
MetaflowHuman-friendly lib that helps scientists and engineers build and manage data science projects
MLRun1,458almost 2 years agoGeneric mechanism for data scientists to build, run, and monitor ML tasks and pipelines
Orchest4,091over 3 years agoVisual pipeline editor and workflow orchestrator with an easy to use UI and based on Kubernetes
Ploomber3,530about 2 years agoWrite maintainable, production-ready pipelines. Develop locally, deploy to the cloud
PrefectA workflow management system, designed for modern infrastructure
VDP2,181almost 2 years agoAn open-source tool to seamlessly integrate AI for unstructured data into the modern data stack
WordwareA web-hosted IDE where non-technical domain experts can build task-specific AI agents
ZenML4,261almost 2 years agoAn extensible open-source MLOps framework to create reproducible pipelines

Resources / Articles

A Tour of End-to-End Machine Learning Platforms(Databaseline)
Continuous Delivery for Machine Learning(Martin Fowler)
Machine Learning Operations (MLOps): Overview, Definition, and Architecture(arXiv)
MLOps Roadmap: A Complete MLOps Career Guide(Scaler Blogs)
MLOps: Continuous delivery and automation pipelines in machine learning(Google)
MLOps: Machine Learning as an Engineering Discipline(Medium)
Rules of Machine Learning: Best Practices for ML Engineering(Google)
The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction(Google)
What Is MLOps?(NVIDIA)

Resources / Books

Beginning MLOps with MLFlow(Apress)
Building Machine Learning Pipelines(O'Reilly)
Building Machine Learning Powered Applications(O'Reilly)
Deep Learning in Production(AI Summer)
Designing Machine Learning Systems(O'Reilly)
Engineering MLOps(Packt)
Implementing MLOps in the Enterprise(O'Reilly)
Introducing MLOps(O'Reilly)
Kubeflow for Machine Learning(O'Reilly)
Kubeflow Operations Guide(O'Reilly)
Machine Learning Design Patterns(O'Reilly)
Machine Learning Engineering in Action(Manning)
ML Ops: Operationalizing Data Science(O'Reilly)
MLOps Engineering at Scale(Manning)
MLOps Lifecycle Toolkit(Apress)
Practical Deep Learning at Scale with MLflow(Packt)
Practical MLOps(O'Reilly)
Production-Ready Applied Deep Learning(Packt)
Reliable Machine Learning(O'Reilly)
The Machine Learning Solutions Architect Handbook(Packt)

Resources / Events

apply() - The ML data engineering conference
MLOps Conference - Keynotes and Panels
MLOps World: Machine Learning in Production Conference
NormConf - The Normcore Tech Conference
Stanford MLSys Seminar Series

Resources / Other Lists

Applied ML27,407about 2 years ago
Awesome AutoML Papers4,035over 2 years ago
Awesome AutoML870almost 2 years ago
Awesome Data Science25,352almost 2 years ago
Awesome DataOps163almost 2 years ago
Awesome Deep Learning24,435over 2 years ago
Awesome Game Datasets763almost 2 years ago(includes AI content)
Awesome Machine Learning66,380almost 2 years ago
Awesome MLOps12,684almost 2 years ago
Awesome Production Machine Learning17,721almost 2 years ago
Awesome Python227,763about 2 years ago
Deep Learning in Production4,313almost 2 years ago

Resources / Podcasts

Kubernetes Podcast from Google
Machine Learning – Software Engineering Daily
MLOps.community
Pipeline Conversation
Practical AI: Machine Learning, Data Science
This Week in Machine Learning & AI
True ML Talks

Resources / Slack

Kubeflow Workspace
MLOps Community Wokspace

Resources / Websites

Feature Stores for ML
Made with ML37,816about 2 years ago
ML-Ops
MLOps Community
MLOps Guide
MLOps Now

Backlinks from these awesome lists:

More related projects: