Adjusted Measures for Feature Selection Stability for Data Sets with Similar Features

Bommert, Andrea; Rahnenführer, Jörg

doi:10.1007/978-3-030-64583-0_19

Cited by 14 publications

(39 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$\textrm{sim}(X_k, X_l)$\end{document} be the similarity of two features \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$X_k$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$X_l$\end{document} , assessed with a similarity measure that attains values in the interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$[0, 1]$\end{document} , for example the absolute Pearson correlation, and let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$\theta \in [0, 1]$\end{document} be a threshold. The stability measure SMA-Count [ 65 ] is defined as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}\begin{align*} &\text{SMA-Count} = \frac{2}{m (m-1)} \sum\limits_{i=1}^{m-1} \sum\limits_{j = i+1}^m S(V_i, V_j) \quad \textrm{with}\\ &S(V_i, V_j) = \frac{\left| V_i \cap V_j \right| \! + \!…”

Section: Methodsmentioning

confidence: 99%

“…The maximum value of SMA-Count is 1 and indicates a perfectly stable feature selection. The stability measure SMA-Count is suitable for data sets that contain highly correlated features such as gene expression data [ 41 ].…”

Section: Methodsmentioning

confidence: 99%

“…Embedded methods require the use of a certain predictive model. Most filter methods, however, are fast to calculate and can be combined with any kind of predictive method, even methods with embedded feature selection, see [ 40 , 41 ]. Also, they can heavily reduce the run time for fitting the subsequent model.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Benchmark of filter methods for feature selection in high-dimensional gene expression survival data

Bommert

Welchowski

Schmid

et al. 2021

Briefings in Bioinformatics

Self Cite

View full text Add to dashboard Cite

Feature selection is crucial for the analysis of high-dimensional data, but benchmark studies for data with a survival outcome are rare. We compare 14 filter methods for feature selection based on 11 high-dimensional gene expression survival data sets. The aim is to provide guidance on the choice of filter methods for other researchers and practitioners. We analyze the accuracy of predictive models that employ the features selected by the filter methods. Also, we consider the run time, the number of selected features for fitting models with high predictive accuracy as well as the feature selection stability. We conclude that the simple variance filter outperforms all other considered filter methods. This filter selects the features with the largest variance and does not take into account the survival outcome. Also, we identify the correlation-adjusted regression scores filter as a more elaborate alternative that allows fitting models with similar predictive accuracy. Additionally, we investigate the filter methods based on feature rankings, finding groups of similar filters.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Benchmark of filter methods for feature selection in high-dimensional gene expression survival data

Bommert

Welchowski

Schmid

et al. 2021

Briefings in Bioinformatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…, V m denote m sets of selected features, |V i | the cardinality of set V i , and E [•] the expected value for a random feature selection. The unadjusted stability measure SMU and the adjusted stability measure SMA (originally called SMA-Count in [2]) are defined as…”

Section: Building Blocksmentioning

confidence: 99%

“…Even though X A and X B provide almost the same information, unadjusted measures consider the selection of X B instead of X A (or vice versa) as a lack of stability. Adjusted stability measures on the other hand take into account the similarities between the features but require more time for calculation [2].…”

Section: Introductionmentioning

confidence: 99%

Employing an Adjusted Stability Measure for Multi-Criteria Model Fitting on Data Sets with Similar Features

Bommert,

Rahnenführer,

Lang

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Fitting models with high predictive accuracy that include all relevant but no irrelevant or redundant features is a challenging task on data sets with similar (e.g. highly correlated) features. We propose the approach of tuning the hyperparameters of a predictive model in a multicriteria fashion with respect to predictive accuracy and feature selection stability. We evaluate this approach based on both simulated and real data sets and we compare it to the standard approach of single-criteria tuning of the hyperparameters as well as to the state-of-the-art technique "stability selection". We conclude that our approach achieves the same or better predictive performance compared to the two established approaches. Considering the stability during tuning does not decrease the predictive accuracy of the resulting models. Our approach succeeds at selecting the relevant features while avoiding irrelevant or redundant features. The single-criteria approach fails at avoiding irrelevant or redundant features and the stability selection approach fails at selecting enough relevant features for achieving acceptable predictive accuracy. For our approach, for data sets with many similar features, the feature selection stability must be evaluated with an adjusted stability measure, that is, a measure that considers similarities between features. For data sets with only few similar features, an unadjusted stability measure suffices and is faster to compute.

show abstract

Employing an Adjusted Stability Measure for Multi-criteria Model Fitting on Data Sets with Similar Features

Bommert

Rahnenführer²,

Lang³

2022

Machine Learning, Optimization, and Data Science

View full text Add to dashboard Cite

Adjusted Measures for Feature Selection Stability for Data Sets with Similar Features

Cited by 14 publications

References 12 publications

Benchmark of filter methods for feature selection in high-dimensional gene expression survival data

Benchmark of filter methods for feature selection in high-dimensional gene expression survival data

Employing an Adjusted Stability Measure for Multi-Criteria Model Fitting on Data Sets with Similar Features

Employing an Adjusted Stability Measure for Multi-criteria Model Fitting on Data Sets with Similar Features

Contact Info

Product

Resources

About