Meta-Statistics for Variable Selection: TheRPackageBioMark

Abstract

Biomarker identification is an ever more important topic in the life sciences. With the advent of measurement methodologies based on microarrays and mass spectrometry, thousands of variables are routinely being measured on complex biological samples. Often, the question is what makes two groups of samples different. Classical hypothesis testing suffers from the multiple testing problem; however, correcting for this often leads to a lack of power. In addition, choosing α cutoff levels remains somewhat arbitrary. Also in a regression context, a model depending on few but relevant variables will be more accurate and precise, and easier to interpret biologically.We propose an R package, BioMark, implementing two meta-statistics for variable selection. The first, higher criticism, presents a data-dependent selection threshold for significance, instead of a cookbook value of α = 0.05. It is applicable in all cases where two groups are compared. The second, stability selection, is more general, and can also be applied in a regression context. This approach uses repeated subsampling of the data in order to assess the variability of the model coefficients and selects those that remain consistently important. It is shown using experimental spike-in data from the field of metabolomics that both approaches work well with real data. BioMark also contains functionality for simulating data with specific characteristics for algorithm development and testing.

Probiotic supplementation in trained trotter horses: effect on blood clinical pathology data and urine metabolomic assessed in field

“…For the purpose, the aov function of the R package “stats” ( 7 ) was applied on the parameters expressed as ranks ( 10 ). To circumvent in part the dichotomous application of P values, as suggested by Greenland et al ( 27 ) and Wehrens and Franceschi ( 57 ), a P value trim limit of 0.1 was considered, in agreement with Kang et al ( 32 ).…”

Performance of variable selection methods using stability-based selection

“…Finally, the findings from this study provide new results for variable selection methods, especially for the stability-based variable selection approach, which has not previously been extensively evaluated for LASSO and Elastic Net [5, 12]. The stability-based student t test and VIP scores outperformed Elastic Net and LASSO in all parameter configurations but not when the effect size and the number of variables were large.…”

“…The R package Biomark [5, 6] includes these popular variable selection methods: student t test, Variable Importance in Projection (VIP) scores [7, 8] from Partial Least Squares Regression (PLS-DA) models, Least Absolute Shrinkage and Selection Operator (LASSO) [9], and Elastic Net [10, 11]. Each method has different strengths and weaknesses for identifying significant variables often found in biological data like in metabolomics, and possibly, for modelling them.…”

“…Those variables appearing by chance in a few perturbations will not be a consistent indicator of class differences when the results are averaged, so are not selected. Thus, stability-based selection, like the jackknife [17] approach, perturbs the data to identify those variables that are consistently selected as group difference indicators [5, 12, 13] to improve prediction.…”

VSURF: An R Package for Variable Selection Using Random Forests

“…The R package spikeSlabGAM implements Bayesian variable selection via regularized estimation in additive mixed models (Scheipl, 2011). Dedicated to the biomarker identification in the life sciences, the R package BioMark implements two meta-statistics for variable selection (Wehrens et al, 2012): the first sets a data-dependent selection threshold for significance, which is useful when two groups are compared, and the second, more general one, uses repeated subsampling and selects the model coefficients remaining consistently important.…”