awesome-AutoML
by windmaple
Curating a list of AutoML-related research, tools, projects and other resources
AI summary
AutoML resource collection
A curated list of AutoML tools, research, projects and resources to aid in the development and exploration of automated machine learning systems.
- stars
- 870
- forks
- 146
- watching
- 43
- awesome list
- 1
- entries
- 288
What's in the list
288 links in 32 sections, with live GitHub stats.activeno commit in 2y
Research papers / AutoML survey
- Neural architecture search: a survey 深度神经网络结构搜索综述
(Tang et al. 2021)
- AutoML to Date and Beyond: Challenges and Opportunities
(Santu et al. 2020)
- Benchmark and Survey of Automated Machine Learning Frameworks
(Zoller et al. 2019)
- AutoML: A Survey of the State-of-the-Art
(He et al. 2019)
- A Survey on Neural Architecture Search
(Wistuba et al. 2019)
- Neural Architecture Search: A Survey
(Elsken et al. 2019)
Research papers / Neural Architecture Search
- Archon: An Architecture Search Framework for Inference-Time Techniques
(Saad-Falcon. 2024)
- LayerNAS: Neural Architecture Search in Polynomial Complexity
(Fan et al. 2023)
- EvoPrompting: Language Models for Code-Level Neural Architecture Search
(Chen et al. 2023)
- Neural Architecture Search using Property Guided Synthesis
(Jin et al. 2022)
- Data-Free Neural Architecture Search via Recursive Label Calibration
(Liu et al. 2022)
- Resource-Constrained Neural Architecture Search on Tabular Datasets
(Yang et al. 2022)
- Searching for Fast Model Families on Datacenter Accelerators
(Li et al. 2022)
- Towards the co-design of neural networks and accelerators
(Zhou et al. 2022)
- Neural Architecture Search for Energy Efficient Always-on Audio Models
(Speckhard et al. 2022)
- KNAS: Green Neural Architecture Search
(Xu et al. 2021)
- AlphaNet: Improved Training of Supernets with Alpha-Divergence
(Wang et al. 2021)
- Speedy Performance Estimation for Neural Architecture Search
(Ru et al. 2021)
- AutoFormer: Searching Transformers for Visual Recognition
(Chen et al. 2021)
- NAAS: Neural Accelerator Architecture Search
(Lin et al. 2021)
- AutoSpace: Neural Architecture Search with Less Human Interference
(Zhou et al. 2021)
- ReNAS:Relativistic Evaluation of Neural Architecture Search
(Xu et al. 2021)
- Searching for Fast Model Families on Datacenter Accelerators
(Li et al. 2021)
- Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
(Lin et al. 2021)
- PyGlove: Symbolic Programming for Automated Machine Learning
(Peng et al. 2021)
- NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
(Chen et al. 2020)
- Few-shot Neural Architecture Search
(Zhao et al. 2020)
- Efficient Neural Architecture Search via Proximal Iterations
(Yao et al. 2020)
- How Does Supernet Help in Neural Architecture Search?
(Zhang et al. 2020)
- MCUNet: Tiny Deep Learning on IoT Devices
(Lin et al. 2020)
- Neural Architecture Transfer
(Lu et al. 2020)
- Semi-Supervised Neural Architecture Search
(Luo et al. 2020)
- MixPath: A Unified Approach for One-shot Neural Architecture Search
(Chu et al. 2020)
- AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
(Real et al. 2020)
- CARS: Continuous Evolution for Efficient Neural Architecture Search
(Yang et al. 2019)
- Meta-Learning of Neural Architectures for Few-Shot Learning
(Elsken et al. 2019)
- Efficient Forward Architecture Search
(Hue et al. 2019)
- Towards Oracle Knowledge Distillation with Neural Architecture Search
(Kang et al. 2019)
- NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
(Ghiasi et al. 2019)
- Evaluating the Search Phase of Neural Architecture Search
(Sciuto et al. 2019)
- MixConv: Mixed Depthwise Convolutional Kernels
(Tan et al. 2019)
- Multinomial Distribution Learning for Effective Neural Architecture Search
(Zheng et al. 2019)
- AutoGAN: Neural Architecture Search for Generative Adversarial Networks
(Gong et al. 2019)
- MixConv: Mixed Depthwise Convolutional Kernels
(Tan et al. 2019)
- Tiny Video Networks
(Piergiovanni et al. 2019)
- MoGA: Searching Beyond MobileNetV3
(Chu et al. 2019) -
- Searching for MobileNetV3
(Howard et al. 2019)
- DetNAS: Backbone Search for Object Detection
(Chen et al. 2019)
- Graph HyperNetworks for Neural Architecture Search
(Zhang et al. 2019)
- Dynamic Distribution Pruning for Efficient Network Architecture Search
(Zheng et al. 2019)
- SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers
(Fedorov et al. 2019)
- EENA: Efficient Evolution of Neural Architecture
(Zhu et al. 2019)
- InstaNAS: Instance-aware Neural Architecture Search
(Cheng et al. 2019)
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
(Ying et al. 2019)
- Evolutionary Neural AutoML for Deep Learning
(Liang et al. 2019)
- The Evolved Transformer
(So et al. 2019)
- SNAS: Stochastic Neural Architecture Search
(Xie et al. 2019)
- NeuNetS: An Automated Synthesis Engine for Neural Network Design
(Sood et al. 2019)
- Understanding and Simplifying One-Shot Architecture Search
(Bender et al. 2018)
- Evolving Space-Time Neural Architectures for Videos
(Piergiovanni et al. 2018)
- IRLAS: Inverse Reinforcement Learning for Architecture Search
(Guo et al. 2018)
- Neural Architecture Search with Bayesian Optimisation and Optimal Transport
(Kandasamy et al. 2018)
- Path-Level Network Transformation for Efficient Architecture Search
(Cai et al. 2018)
- BlockQNN: Efficient Block-wise Neural Network Architecture Generation
(Zhong et al. 2018)
- Stochastic Adaptive Neural Architecture Search for Keyword Spotting
(Véniat et al. 2018)
- Task-Driven Convolutional Recurrent Models of the Visual System
(Nayebi et al. 2018)
- Neural Architecture Optimization
(Luo et al. 2018)
- MnasNet: Platform-Aware Neural Architecture Search for Mobile
(Tan et al. 2018)
- Neural Architecture Search: A Survey
(Elsken et al. 2018)
- Auto-Meta: Automated Gradient Based Meta Learner Search
(Kim et al. 2018)
- MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
(Gordon et al. 2018)
- Searching Toward Pareto-Optimal Device-Aware Neural Architectures
(Cheng et al. 2018)
- Differentiable Architecture Search
(Liu et al. 2018)
- Regularized Evolution for Image Classifier Architecture Search
(Real et al. 2018)
- Efficient Architecture Search by Network Transformation
(Cai et al. 2017)
- Large-Scale Evolution of Image Classifiers
(Real et al. 2017)
- Progressive Neural Architecture Search
(Liu et al. 2017)
- AdaNet: Adaptive Structural Learning of Artificial Neural Networks
(Cortes et al. 2017)
- Learning Transferable Architectures for Scalable Image Recognition
(Zoph et al. 2017)
Research papers / Federated Neural Architecture Search
- Federated Neural Architecture Search
(Xu et al 2020)
- Direct Federated Neural Architecture Search
(Garg et al 2020)
Research papers / Neural Architecture Search benchmark
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
(Ying et al. 2019) -
Research papers / Neural Optimizatizer Search
- Symbolic Discovery of Optimization Algorithms
(Chen et al. 2017)
- Neural Optimizer Search with Reinforcement Learning
(Bello et al. 2017)
Research papers / Activation function Search
- Searching for Activation Functions
(Ramachandran et al. 2017)
Research papers / AutoAugment
- MetaAugment: Sample-Aware Data Augmentation Policy Learning
(Zhou et al. 2020)
- Learning Data Augmentation Strategies for Object Detection
(Zoph et al. 2019)
- Fast AutoAugment
(Lim et al. 2019)
- AutoAugment: Learning Augmentation Policies from Data
(Cubuk et al. 2018)
Research papers / AutoDropout
- AutoDropout: Learning Dropout Patterns to Regularize Deep Networks
(Pham et al. 2020)
Research papers / AutoDistill
Research papers / Learning to learn/Meta-learning
- ES-MAML: Simple Hessian-Free Meta Learning
(Song et al. 2019)
- Learning to Learn with Gradients
(Chelsea Finn PhD disseration 2018)
- On First-Order Meta-Learning Algorithms
(OpenAI Reptile by Nichol et al. 2018)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
(MAML by Finn et al. 2017)
- A sample neural attentive meta-learner
(Mishra et al. 2017)
- Learning to Learn without Gradient Descent by Gradient Descent
(Chen et al. 2016)
- Learning to learn by gradient descent by gradient descent
(Andrychowicz et al. 2016)
- Learning to reinforcement learn
(Wang et al. 2016)
- RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning
(Duan et al. 2016)
Research papers / Hyperparameter optimization
- Frugal Optimization for Cost-related Hyperparameters
(Qingyun Wu, Chi Wang, Silu Huang. AAAI 2021)
- Economical Hyperparameter Optimization With Blended Search Strategy
(Chi Wang, Qingyun Wu, Silu Huang, Amin Saied. ICLR 2021)
- ChaCha for Online AutoML
(Qingyun Wu, Chi Wang, John Langford, Paul Mineiro and Marco Rossi. ICML 2021)
- Population Based Training of Neural Networks
(Jaderberg et al. 2017)
- Google Vizier: A Service for Black-Box Optimization
(Golovin et al. 2017)
- Practical Bayesian Optimization of Machine Learning Algorithms
(Snoek et al. 2012)
- Random Search for Hyper-Parameter Optimization
(Bergstra et al. 2012)
Research papers / Automatic feature selection
- Deep Feature Synthesis: Towards Automating Data Science Endeavors
(Kanter et al. 2017)
- ExploreKit: Automatic Feature Generation and Selection
(Katz et al. 2016)
Research papers / Recommendation systems
- AutoML for Deep Recommender Systems: A Survey
(Zheng et al. 2022)
- Automated Machine Learning for Deep Recommender Systems: A Survey
(Chen et al. 2022)
- AutoDim: Field-aware Embedding Dimension Searchin Recommender Systems
(Zhao et al. 2021)
- Learnable Embedding Sizes for Recommender Systems
(Liu et al. 2021)
- AIM: Automatic Interaction Machine for Click-Through Rate Prediction
(Zhu et al. 2021)
- Neural Input Search for Large Scale Recommendation Models
(Joglekar et al. 2019)
Research papers / Model compression
Research papers / Quantization
- HAQ: Hardware-Aware Automated Quantization with Mixed Precision
(Wang et al. 2018)
Research papers / Tech to speech
Research papers / Bandits
- AutoML for Contextual Bandits
(Dutta et al. 2019)
Research papers / Reinforcement learning
- Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
(Parker-Holder et al. 2022)
- Designing Neural Network Architectures using Reinforcement Learning
(Baker et al. 2016)
- Neural Architecture Search with Reinforcement Learning
(Zoph and Le. 2016)
Research papers / Graph neural network
- AutoGL: A Library for Automated Graph Learning
(Guan et al. 2021)
- AutoGraph: Automated Graph Neural Network
(Li et al. 2020)
Research papers / Quantum computing
- QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits
(Wang et al. 2022)
- Differentiable Quantum Architecture Search
(Zhang et al. 2020)
Research papers / Prompt search
- Large Language Models Are Human-Level Prompt Engineers
(Zhou et al. 2022)
- Neural Prompt Search
(Zhang et al. 2022)
Research papers / LLM
- LLaMA-NAS: Efficient Neural Architecture Search for Large Language Models
(Sarah et al. 2024)
Tools and projects
Falcon
: A Lightweight AutoML Library
MindWare
: Efficient Open-source AutoML System
AutoDL
: automated deep learning
AutoGL
: An autoML framework & toolkit for machine learning on graphs
MLBox
: a powerful Automated Machine Learning python library
FLAML
: Fast and lightweight AutoML ( )
Hypernets
: A General Automated Machine Learning Framework
Cooka
: a lightweight and visualization toolkit
Vegas
: an AutoML algorithm tool chain by Huawei Noah's Arb Lab
TransmogrifAI
: an AutoML library written in Scala that runs on top of Apache Spark
Model Search
: a framework that implements AutoML algorithms for model architecture search at scale
- AutoGluon
: AutoML Toolkit for Deep Learning
hyperunity
: A toolset for black-box hyperparameter optimisation
auptimizer
: An automatic ML model optimization tool
Keras Tuner
: Hyperparameter tuning for humans
Torchmeta
: A Meta-Learning library for PyTorch
learn2learn
: PyTorch Meta-learning Framework for Researchers
Auto-PyTorch
: Automatic architecture search and hyperparameter optimization for PyTorch
- ATM: Auto Tune Models
: A multi-tenant, multi-data system for automated machine learning (model selection and tuning)
Adanet: Fast and flexible AutoML with learning guarantees
: Tensorflow package for AdaNet
Microsoft Neural Network Intelligence (NNI)
: An open source AutoML toolkit for neural architecture search and hyper-parameter tuning
Dragonfly
: An open source python library for scalable Bayesian optimisation
- H2O AutoML
: Automatic Machine Learning by H2O.ai
Kubernetes Katib
: hyperparameter Tuning on Kubernetes inspired by Google Vizier
- Ray Tune
: Scalable Hyperparameter Tuning¶
- TransmogrifAI
: automated machine learning for structured data by Salesforce
Advisor
: open-source implementation of Google Vizier for hyper parameters tuning
- AutoKeras
: AutoML library by Texas A&M University using Bayesian optimization
- AutoSklearn
: an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator
Ludwig
: a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code
- AutoWeka
: hyperparameter search for Weka
automl-gs
: Provide an input CSV and a target field to predict, generate a model + code to run it
SMAC
: Sequential Model-based Algorithm Configuration
Hyperopt-sklearn
: hyper-parameter optimization for sklearn
Spearmint
: a software package to perform Bayesian optimization
- TPOT
: one of the very first AutoML methods and open-source software packages
MOE
: a global, black box optimization engine for real world metric optimization by Yelp
Hyperband
: open source code for tuning hyperparams with Hyperband
- Optuna
: define-by-run hypterparameter optimization framework
RoBO
: a Robust Bayesian Optimization framework
HpBandSter
: a framework for distributed hyperparameter optimization
HPOlib2
: a library for hyperparameter optimization and black box optimization benchmarks
- Hyperopt
: distributed Asynchronous Hyperparameter Optimization in Python
REMBO
: Bayesian optimization in high-dimensions via random embedding
- ExploreKit
: a framework for automated feature generation
FeatureTools
: An open source python framework for automated feature engineering
EvalML
: An open source python library for AutoML
PocketFlow
: use AutoML to do model compression (open sourced by Tencent)
DEvol (DeepEvolution)
: a basic proof of concept for genetic architecture search in Keras
mljar-supervised
: AutoML with explanations and markdown reports
Determined
: scalable deep learning training platform with integrated hyperparameter tuning support; includes Hyperband, PBT, and other search methods
AutoGL
: an autoML framework & toolkit for machine learning on graphs)
FEDOT
: AutoML framework for the design of composite pipelines
NASGym
: a proof-of-concept OpenAI Gym environment for Neural Architecture Search (NAS)
Archai
: a platform for Neural Network Search (NAS) that allows you to generate efficient deep networks for your applications
autoBOT
: An autoML system for automated text classification exploiting representation evolution
autoai
: A framework to find the best performing AI/ML model for any AI problem
Benchmarks
Commercial products
- deci.ai AutoNAC
: Automated Neural Architecture Construction (AutoNAC™)
- Databricks AutoML
: Augment experts. Empower citizen data scientists
- Abacus.AI
: Effortlessly Embed Cutting-Edge AI Into Your Apps
Syne Tune
: state-of-the-art distributed hyperparameter optimizers (HPO)
Blog posts
Courses
- https://www.youtube.com/watch?v=EFpGQoDQ7JI
[EfficientML.ai Lecture 8 - Neural Architecture Search (Part II) (MIT 6.5940, Fall 2023)( )
Presentations
- Automatic Machine Learning
by Frank Hutter and Joaquin Vanschoren
- Advanced Machine Learning Day 3: Neural Architecture Search
by Debadeepta Dey (MSR)
- Neural Architecture Search
by Quoc Le (Google Brain)
Books
- Automated Machine Learning in Action
A book that introduces autoML with AutoKreas and Keras Tuner
Competitions, workshops and conferences
Other curated resources on AutoML
Practical applications
Nothing in this list matches your filter.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions awesome-AutoML.
More related projects
microsoft/mmdnn5.8K
microsoft/pai2.6K
microsoft/frameworkcontroller174
liyanghart/hyperparameter-optimization-of-machine-learning-algorithms1.3K
western-oc2-lab/automl-implementation-for-static-and-dynamic-data-analytics624
hyperopt/hyperopt7.3K
optuna/optuna11.1K
scikit-optimize/scikit-optimize2.7K
dssg/aequitas701
parrt/dtreeviz3K
microsoft/neuronblocks1.4K
esri/deep-learning-frameworks464