AutoFe-Sel: A Meta-learning based methodology for Recommending Feature Subset Selection Algorithms

Abstract

Automated machine learning, often referred to as "AutoML," is the process of automating the time-consuming and iterative procedures that are associated with the building of machine learning models. There have been significant contributions in this area across a number of different stages of accomplishing a data-mining task, including model selection, hyperparameter optimization, and preprocessing method selection. Among them, preprocessing method selection is a relatively new and fast growing research area. The current work is focused on the recommendation of preprocessing methods, i.e., feature subset selection (FSS) algorithms. One limitation in the existing studies regarding FSS algorithm recommendation is the use of a single learner for meta-modeling, which restricts its capabilities in the metamodeling. Moreover, the meta-modeling in the existing studies is typically based on a single group of data characterization measures (DCMs). Nonetheless, there are a number of complementary DCM groups, and their combination will allow them to leverage their diversity, resulting in improved meta-modeling. This study aims to address these limitations by proposing an architecture for preprocess method selection that uses ensemble learning for meta-modeling, namely AutoFE-Sel. To evaluate the proposed method, we performed an extensive experimental evaluation involving 8 FSS algorithms, 3 groups of DCMs, and 125 datasets. Results show that the proposed method achieves better performance compared to three baseline methods. The proposed architecture can also be easily extended to other preprocessing method selections, e.g., noise-filter selection and imbalance handling method selection.

Enhanced prediction of cement raw meal oxides by near-infrared spectroscopy using machine learning combined with chemometric techniques

“…Meta-modeling leverages the collective strengths of multiple independent models to improve overall performance significantly ( Khan et al, 2023 ). This integration, which may involve techniques like weighted averaging or model ensembling, endows the final meta-model with enhanced generalization capabilities.…”

An automated approach for binary classification on imbalanced data

“…Specific works for pre-processing based on meta-learning include noise filter selection [15] and feature selection [16][17][18]. Concerning imbalanced learning, to the extent of our knowledge, only two works addressed the automation of imbalanced learning.…”