Automated machine learning

Machine learning and data mining
Part of a series on
Paradigms Supervised learning Unsupervised learning Online learning Batch learning Meta-learning Semi-supervised learning Self-supervised learning Reinforcement learning Rule-based learning Quantum machine learning
Problems Classification Generative model Regression Clustering dimension reduction density estimation Anomaly detection Data Cleaning AutoML Association rules Semantic analysis Structured prediction Feature engineering Feature learning Learning to rank Grammar induction Ontology learning Multimodal learning
Supervised learning (classification • regression) Apprenticeship learning Decision trees Ensembles Bagging Boosting Random forest k-NN Linear regression Naive Bayes Artificial neural networks Logistic regression Perceptron Relevance vector machine (RVM) Support vector machine (SVM)
Clustering BIRCH CURE Hierarchical k-means Fuzzy Expectation–maximization (EM) DBSCAN OPTICS Mean shift
Dimensionality reduction Factor analysis CCA ICA LDA NMF PCA PGD t-SNE SDL
Structured prediction Graphical models Bayes net Conditional random field Hidden Markov
Anomaly detection RANSAC k-NN Local outlier factor Isolation forest
Artificial neural network Autoencoder Cognitive computing Deep learning DeepDream Feedforward neural network Recurrent neural network LSTM GRU ESN reservoir computing Restricted Boltzmann machine GAN Diffusion model SOM Convolutional neural network U-Net Transformer Vision Mamba Spiking neural network Memtransistor Electrochemical RAM (ECRAM)
Reinforcement learning Q-learning SARSA Temporal difference (TD) Multi-agent Self-play
Learning with humans Active learning Crowdsourcing Human-in-the-loop
Model diagnostics Coefficient of determination Confusion matrix Learning curve ROC curve
Mathematical foundations Kernel machines Bias–variance tradeoff Computational learning theory Empirical risk minimization Occam learning PAC learning Statistical learning VC theory
Machine-learning venues ECML PKDD NeurIPS ICML ICLR IJCAI ML JMLR
Related articles Glossary of artificial intelligence List of datasets for machine-learning research List of datasets in computer vision and image processing Outline of machine learning
v t e

Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems.

AutoML potentially includes every stage from beginning with a raw dataset to building a machine learning model ready for deployment. AutoML was proposed as an artificial intelligence-based solution to the growing challenge of applying machine learning.^[1]^[2] The high degree of automation in AutoML aims to allow non-experts to make use of machine learning models and techniques without requiring them to become experts in machine learning. Automating the process of applying machine learning end-to-end additionally offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform hand-designed models.^[3]

Common techniques used in AutoML include hyperparameter optimization, meta-learning and neural architecture search.

YouTube Encyclopedic

1/5
Views:
262 817
18 521
3 905
19 853
426

Transcription

Comparison to the standard approach

In a typical machine learning application, practitioners have a set of input data points to be used for training. The raw data may not be in a form that all algorithms can be applied to. To make the data amenable for machine learning, an expert may have to apply appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods. After these steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model. If deep learning is used, the architecture of the neural network must also be chosen by the machine learning expert.

Each of these steps may be challenging, resulting in significant hurdles to using machine learning. AutoML aims to simplify these steps for non-experts, and to make it easier for them to use machine learning techniques correctly and effectively.

AutoML plays an important role within the broader approach of automating data science, which also includes challenging tasks such as data engineering, data exploration and model interpretation and prediction.^[4]

Targets of automation

Automated machine learning can target various stages of the machine learning process.^[2] Steps to automate are:

Data preparation and ingestion (from raw data and miscellaneous formats)
- Column type detection; e.g., boolean, discrete numerical, continuous numerical, or text
- Column intent detection; e.g., target/label, stratification field, numerical feature, categorical text feature, or free text feature
- Task detection; e.g., binary classification, regression, clustering, or ranking
Feature engineering
- Feature selection
- Feature extraction
- Meta-learning and transfer learning
- Detection and handling of skewed data and/or missing values
Model selection - choosing which machine learning algorithm to use, often including multiple competing software implementations
Ensembling - a form of consensus where using multiple models often gives better results than any single model^[5]
Hyperparameter optimization of the learning algorithm and featurization
Pipeline selection under time, memory, and complexity constraints
Selection of evaluation metrics and validation procedures
Problem checking
- Leakage detection
- Misconfiguration detection
Analysis of obtained results
Creating user interfaces and visualizations

Challenges and Limitations

There are a number of key challenges being tackled around automated machine learning. A big issue surrounding the field is referred to as "development as a cottage industry".^[6] This phrase refers to the issue in machine learning where development relies on manual decisions and biases of experts. This is contrasted to the goal of machine learning which is to create systems that can learn and improve from their own usage and analysis of the data. Basically, it's the struggle between how much experts should get involved in the learning of the systems versus how much freedom they should be giving the machines. However, experts and developers must help create and guide these machines to prepare them for their own learning. To create this system, it requires labor intensive work with knowledge of machine learning algorithms and system design.^[7]

Additionally, some other challenges include meta-learning challenges^[8] and computational resource allocation.

References

^ Thornton C, Hutter F, Hoos HH, Leyton-Brown K (2013). Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms. KDD '13 Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 847–855.
^ ^a ^b Hutter F, Caruana R, Bardenet R, Bilenko M, Guyon I, Kegl B, and Larochelle H. "AutoML 2014 @ ICML". AutoML 2014 Workshop @ ICML. Retrieved 2018-03-28.^{[permanent dead link]}
^ Olson, R.S., Urbanowicz, R.J., Andrews, P.C., Lavender, N.A., Kidd, L.C., Moore, J.H. (2016). Automating Biomedical Data Science Through Tree-Based Pipeline Optimization. In: Squillero, G., Burelli, P. (eds) Applications of Evolutionary Computation. EvoApplications 2016. Lecture Notes in Computer Science(), vol 9597. Springer, Cham. doi:10.1007/978-3-319-31204-0_9
^ De Bie, Tijl; De Raedt, Luc; Hernández-Orallo, José; Hoos, Holger H.; Smyth, Padhraic; Williams, Christopher K. I. (March 2022). "Automating Data Science". Communications of the ACM. 65 (3): 76–87. doi:10.1145/3495256. hdl:10251/199907.
^ Erickson, Nick; Mueller, Jonas; Shirkov, Alexander; Zhang, Hang; Larroy, Pedro; Li, Mu; Smola, Alexander (2020-03-13). "AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data". arXiv:2003.06505 [stat.ML].
^ Hutter, Frank; Kotthoff, Lars; Vanschoren, Joaquin, eds. (2019). Automated Machine Learning: Methods, Systems, Challenges. Springer Nature.
^ Glover, Ellen (2018). "Machine Learning with Python: Clustering". Built In. doi:10.4135/9781526466426.
^ "Meta Learning Challenges". metalearning.chalearn.org. Retrieved 2023-12-03.

From Wikipedia, the free encyclopedia