Cost-Constrained feature selection in binary classification: adaptations for greedy forward selection and genetic algorithms

Abstract

Background: With modern methods in biotechnology, the search for biomarkers has advanced to a challenging statistical task exploring high dimensional data sets. Feature selection is a widely researched preprocessing step to handle huge numbers of biomarker candidates and has special importance for the analysis of biomedical data. Such data sets often include many input features not related to the diagnostic or therapeutic target variable. A less researched, but also relevant aspect for medical applications are costs of different biomarker candidates. These costs are often financial costs, but can also refer to other aspects, for example the decision between a painful biopsy marker and a simple urine test. In this paper, we propose extensions to two feature selection methods to control the total amount of such costs: greedy forward selection and genetic algorithms. In comprehensive simulation studies of binary classification tasks, we compare the predictive performance, the run-time and the detection rate of relevant features for the new proposed methods and five baseline alternatives to handle budget constraints. Results: In simulations with a predefined budget constraint, our proposed methods outperform the baseline alternatives, with just minor differences between them. Only in the scenario without an actual budget constraint, our adapted greedy forward selection approach showed a clear drop in performance compared to the other methods. However, introducing a hyperparameter to adapt the benefit-cost trade-off in this method could overcome this weakness. Conclusions: In feature cost scenarios, where a total budget has to be met, common feature selection algorithms are often not suitable to identify well performing subsets for a modelling task. Adaptations of these algorithms such as the ones proposed in this paper can help to tackle this problem.

Implications on Feature Detection When Using the Benefit–Cost Ratio

“…To avoid this problem, we recommend using an adapted benefit-cost ratio, such as the ones proposed in Jagdhuber et al [7] or Min et al [13]. The main alternative solution to incorporate costs is a weighted linear combination as mentioned in the introduction of this paper.…”

“…Min et al [13,14] presents cost-sensitive feature selection heuristics and also provide a thorough problem definition in the context of rough sets. Jagdhuber et al [7] and Liu et al [11] further extend this idea and propose genetic algorithms with fixed feature cost budgets. For situations without a fixed cost limit, the goal may be to harmonize costs of features and costs of prediction errors by identifying an optimal trade-off.…”

Selection of Summary Statistics for Network Model Choice with Approximate Bayesian Computation

“…A recent wrapper proposal from Zhang et al [2019] used an artificial bee colony algorithm for subset exploration. When imposing a total cost limitation (a.k.a., hard-margin), Jagdhuber et al [2020] proposed a modified genetic algorithm as well as a greedy forward selection approach based on the Akaike Information Criterion. Our goal with such methods was to select a subset of summaries with low computational cost that do not alter the accuracy of the classifier compared to the version that ignores cost.…”

Budget constrained machine learning for early prediction of adverse outcomes for COVID-19 patients

“…These works have tended to provide approximate methods or heuristics for solving the cost-constrained optimization problems associated with such tasks 14 – 17 . In healthcare applications, a number of previous works have introduced budget constraints, such as the financial costs of lab tests or clinical preferences, into their proposed machine learning models 18 , 19 . However, to the best of our knowledge, our work is the first to consider cost-sensitive learning applied to COVID-19 .…”