Eight years of AutoML: categorisation, review and trends

Abstract

Knowledge extraction through machine learning techniques has been successfully applied in a large number of application domains. However, apart from the required technical knowledge and background in the application domain, it usually involves a number of time-consuming and repetitive steps. Automated machine learning (AutoML) emerged in 2014 as an attempt to mitigate these issues, making machine learning methods more practicable to both data scientists and domain experts. AutoML is a broad area encompassing a wide range of approaches aimed at addressing a diversity of tasks over the different phases of the knowledge discovery process being automated with specific techniques. To provide a big picture of the whole area, we have conducted a systematic literature review based on a proposed taxonomy that permits categorising 447 primary studies selected from a search of 31,048 papers. This review performs an extensive and rigorous analysis of the AutoML field, scrutinising how the primary studies have addressed the dimensions of the taxonomy, and identifying any gaps that remain unexplored as well as potential future trends. The analysis of these studies has yielded some intriguing findings. For instance, we have observed a significant growth in the number of publications since 2018. Additionally, it is noteworthy that the algorithm selection problem has gradually been superseded by the challenge of workflow composition, which automates more than one phase of the knowledge discovery process simultaneously. Of all the tasks in AutoML, the growth of neural architecture search is particularly noticeable.

Grammar-based evolutionary approach for automated workflow composition with domain-specific operators and ensemble diversity

“…AWC involves optimising three related dimensions: (1) algorithms, (2) their relationships, and, optionally, (3) their hyper-parameters values. It is important to note that the algorithm selection problem [28] specifically refers to recommending the best algorithm(s) for a given dataset and phase of the knowledge discovery process [5]. On the other hand, AWC provides more comprehensive support by covering multiple phases.…”

“…A notable example is the automatic selection of the most appropriate algorithm for model building [4]. Recently, this idea of automating ML tasks has been formalised in the area of Automated Machine Learning (AutoML) [5]. AutoML provides data scientists with a broader range of alternatives, enabling them to focus on phases that require their expertise and intuition, ultimately bridging the gap between knowledge discovery and domain experts [6,7].…”

“…To alleviate these shortcomings, some authors have proposed automating workflow composition, involving two or more phases of the knowledge discovery process [14,15]. Automated workflow composition (AWC) refers to the process of finding the sequence of data processing steps, which typically include data preprocessing, feature selection, and machine learning algorithms, optionally tuning their hyper-parameters, that provides the best performance for a particular machine learning (ML) task (e.g., classification) [5]. AWC represents a challenging task [16] that is commonly approached as an optimisation problem aiming at constructing a workflow that optimally prepares and analyses a dataset.…”

A Review of Machine Learning Techniques in Agroclimatic Studies

“…AutoML bridges the gap between complex models and practical agricultural applications, offering scalable solutions to food security and sustainability challenges. By automating the ML workflow, including algorithm selection and hyperparameter tuning, AutoML simplifies ML deployment in agriculture [11,42]. This extends from data preprocessing to model evaluation, highlighting its potential to democratize ML applications in the sector.…”

Throughput Time Predictions Along the Order Fulfilment Process

“…This topic is titled as "AutoML", see e.g. [31][32][33]). These automate the selection of a suitable algorithm, (e.g.…”