Automated machine learning

Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems. AutoML potentially includes every stage from beginning with a raw dataset to building a machine learning model ready for deployment. AutoML was proposed as an artificial intelligence-based solution to the growing challenge of applying machine learning.[1][2] The high degree of automation in AutoML aims to allow non-experts to make use of machine learning models and techniques without requiring them to become experts in machine learning. Automating the process of applying machine learning end-to-end additionally offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform hand-designed models. Common techniques used in AutoML include hyperparameter optimization, meta-learning and neural architecture search.

Part of a series on
Machine learning
and data mining
Problems
Supervised learning
(classification • regression)
Clustering
Dimensionality reduction
Structured prediction
Anomaly detection
Artificial neural network
Reinforcement learning
Theory
Machine-learning venues
Related articles

Comparison to the standard approach

In a typical machine learning application, practitioners have a set of input data points to be used for training. The raw data may not be in a form that all algorithms can be applied to. To make the data amenable for machine learning, an expert may have to apply appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods. After these steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model. If deep learning is used, the architecture of the neural network must also be chosen by the machine learning expert.

Each of these steps may be challenging, resulting in significant hurdles to using machine learning. AutoML aims to simplify these steps for non-experts, and make the practice of machine learning more efficient.

Targets of automation

Automated machine learning can target various stages of the machine learning process.[2] Steps to automate are:

AutoML Systems

In recent years, many AutoML frameworks have been developed, which target both researchers and end users. We provide here some examples of these AutoML packages:

  • Auto-WEKA[1][4] is a machine learning package built on WEKA, which considers joint algorithm selection and hyperparameter optimization in addition to preprocessing steps.
  • Auto-sklearn[5] is a drop-in replacement for regular scikit-learn classifiers and regressors, and uses Bayesian optimization to find good machine learning pipelines.
  • H2O AutoML provides an easy-to-use interface which automates the process of training and tuning a large selection of candidate models. This is based on the H2O platform which provides unified interfaces to a variety of machine learning algorithms.
  • TPOT[6][7] is an Automated Machine Learning tool that uses scikit-learn algorithms to optimize machine learning pipelines.
  • AutoGluon is an AutoML system which enables the user to prototype machine learning models for raw text, image, and tabular data, by leveraging automatic hyperparameter tuning, model selection/ensembling, architecture search, and data processing.
  • MLBox is an Automated Machine Learning library which provides data preprocessing, hyper-parameter optimization, and state-of-the art predictive models for classification and regression.
  • TransmogrifAI is a library built on Scala and SparkML that automates data cleansing, feature engineering, and model selection.
  • Auto-PyTorch[8][9] is an AutoML framework based on PyTorch which jointly optimizes the neural network architecture and the training hyperparameters.
  • Auto-Keras is an AutoML system which performs efficient neural architecture search by morphing the neural architectures guided by Bayesian optimization[10].
  • Talos is a package which fully automates hyperparameter tuning and model evaluation with TensorFlow (tf.keras) and PyTorch.

See also

References
  1. Thornton C, Hutter F, Hoos HH, Leyton-Brown K (2013). Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms. KDD '13 Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 847–855.
  2. Hutter F, Caruana R, Bardenet R, Bilenko M, Guyon I, Kegl B, and Larochelle H. "AutoML 2014 @ ICML". AutoML 2014 Workshop @ ICML. Retrieved 2018-03-28.
  3. Erickson, Nick; Mueller, Jonas; Shirkov, Alexander; Zhang, Hang; Larroy, Pedro; Li, Mu; Smola, Alexander (2020-03-13). "AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data". arXiv:2003.06505 [stat.ML].
  4. Lars Kotthoff, Chris Thornton, Holger Hoos, Frank Hutter, and Kevin Leyton-Brown (2017). Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA JMLR. 18(25):1−5, 2017
  5. Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum and Frank Hutter (2015). Efficient and Robust Automated Machine Learning Advances in Neural Information Processing Systems 28 (NIPS 2015).
  6. Randal S. Olson, Ryan J. Urbanowicz, Peter C. Andrews, Nicole A. Lavender, La Creis Kidd, and Jason H. Moore (2016). Automating biomedical data science through tree-based pipeline optimization. Applications of Evolutionary Computation, pages 123-137.
  7. Randal S. Olson, Nathan Bartley, Ryan J. Urbanowicz, and Jason H. Moore (2016).Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science.Proceedings of GECCO 2016, pages 485-492.
  8. Zimmer, L., Lindauer, M. and Hutter, F., 2021. Auto-Pytorch: multi-fidelity metalearning for efficient and robust autoDL. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9), pp.3079-3090.
  9. Mendoza, H., Klein, A., Feurer, M., Springenberg, J. T., Urban, M., Burkart, M., ... & Hutter, F. (2019). Towards automatically-tuned deep neural networks. In Automated machine learning (pp. 135-149). Springer, Cham.
  10. Jin, H., Song, Q. and Hu, X., 2019, July. Auto-keras: An efficient neural architecture search system. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1946-1956).

Further reading
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.