Automatic Creation of Machine Learning Workflows with Strongly Typed Genetic Programming

Abstract

Manual creation of machine learning ensembles is a hard and tedious task which requires an expert and a lot of time. In this work we describe a new version of the GP-ML algorithm which uses genetic programming to create machine learning workows (combinations of preprocessing, classification, and ensembles) automatically, using strongly typed genetic programming and asynchronous evolution. The current version improves the way in which the individuals in the genetic programming are created and allows for much larger workows. Additionally, we added new machine learning methods. The algorithm is compared to the grid search of the base methods and to its previous versions on a set of problems from the UCI machine learning repository.

Automated Selection and Configuration of Multi-Label Classification Algorithms with Grammar-Based Genetic Programming

“…Currently, Auto-ML methods [3,5,6,8,15] have been dealing with the optimization of complete ML pipelines. This means that, instead of just focusing on ML algorithms and their hyper-parameters, these methods are also concerned with other aspects of ML, such as data preprocessing (e.g., feature normalization or feature selection) and post-processing (e.g., classification probability calibration).…”

“…The Tree-Based Pipeline Optimization Tool (TPOT) [8], for instance, applies a canonical genetic programming (GP) algorithm to search for the most appropriate ML pipeline in the Scikit-Learn library. Considering a different evolutionary approach, the Genetic Programming for Machine Learning method (GP-ML) [6] uses a strongly typed genetic programming (STGP) method to restrict the Scikit-Learn pipelines in such a way that they are always meaningful from the machine learning point of view. Finally, the REsilient ClassifIcation Pipeline Evolution method (RECIPE) [3] adopts a grammar-based genetic programming (GGP) method to search for Scikit-Learn pipelines.…”

“…The outgrowing popularity of machine learning algorithms and its indiscriminate use by practitioners who do not necessarily know the peculiarities of these methods have made the area of automated machine learning (Auto-ML) [3,5,6,8,15] more relevant than ever. The area of Auto-ML emerged to deal with the problem of how to select learning algorithms and their hyper-parameters to successfully solve a given ML problem.…”

“…Auto-ML proposes to deal with these problems by customizing solutions (in terms of algorithms and configurations) to ML problems. Most Auto-ML systems proposed to date focus on generating sequences of steps to solve single label classification (SLC) problems [3,5,6,8,15]. The objective of classification is to learn models from data capable of expressing the relationships between a set of predictive attributes and a predefined set of class labels.…”

An extensive experimental evaluation of automated machine learning methods for recommending classification algorithms

“…There are also other recently proposed EAs for related AutoML tasks. In particular, the EAs proposed in [18][19][20] try to optimize an entire machine learning pipeline for a given dataset, including the choice of data preprocessing methods (like feature scaling operators and feature selection methods) and classification algorithm. By contrast, we focus on using EAs that recommend only classification algorithms.…”