A systematic review and evaluation of statistical methods for group variable selection

Abstract

This review condenses the knowledge on variable selection methods implemented in R and appropriate for datasets with grouped features. The focus is on regularized regressions identified through a systematic review of the literature, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. A total of 14 methods are discussed, most of which use penalty terms to perform group variable selection. Depending on how the methods account for the group structure, they can be classified into knowledge and data-driven approaches. The first encompass group-level and bi-level selection methods, while two-step approaches and collinearity-tolerant methods constitute the second category. The identified methods are briefly explained and their performance compared in a simulation study. This comparison demonstrated that group-level selection methods, such as the group minimax concave penalty, are superior to other methods in selecting relevant variable groups but are inferior in identifying important individual variables in scenarios where not all variables in the groups are predictive. This can be better achieved by bi-level selection methods such as group bridge. Two-step and collinearity-tolerant approaches such as elastic net and ordered homogeneity pursuit least absolute shrinkage and selection operator are inferior to knowledge-driven methods but provide results without requiring prior knowledge. Possible applications in proteomics are considered, leading to suggestions on which method to use depending on existing prior knowledge and research question.

Interpretability of bi‐level variable selection methods

“…Since the use of knowledge‐driven model building usually resulted in the highest values for interpretability, this concept seems to be recommendable. This is largely in line with other findings in the literature (Buch et al., 2023). However, it must be taken into account that a random group assignment performed in some selection tasks better than the alternative groupings.…”

“…The predictive performance of the methods in simulation studies supports the ranking of selection relevance postulated here, with cMCP at the top (Liu et al., 2013). Comprehensive simulation studies with knowledge‐ and correlation‐based groups have already suggested that GEL tends toward collinearity‐tolerant selection and the inability of the same for SGL and cMCP (Buch et al., 2023). Yet, even if interpretability could be improved, the integration of grouping is not generally advisable over a classical approach.…”

“…A possible starting point to refine the method would be to combine other penalty functions in the style of SGL instead of the group LASSO and the classical LASSO. A mixture of the group MCP (Wei & Zhu, 2012) and the classical MCP has already been presented (Liu et al., 2013), but a suitable implementation of this approach is lacking (Huang et al., 2012; Buch et al., 2023). The technique presented there seems to form more parsimonious models and responds to a change of $$\alpha $$ in a more recognizable way.…”

Integrative polygenic risk score improves the prediction accuracy of complex traits and diseases

“…To combine the scores, we employed Elastic Net 18 to construct linear combinations of the PRS. We proposed two combination frameworks: 1) PRSmix combines the scores developed from the same outcome trait, and 2) PRSmix+ combines all the high-power scores across other traits.…”

Integrative polygenic risk score improves the prediction accuracy of complex traits and diseases

“…To combine the scores, we employed Elastic Net 19 to construct linear combinations of the PRS. We proposed two combination frameworks: (1) PRSmix combines the scores developed from the same outcome trait, and (2) PRSmix+ combines all the high-power scores across other traits.…”