Model selection in high-dimensional noisy data: a simulation study

Abstract

In many practical applications, high-dimensional regression analyses have to take into account measurement error in the covariates. It is thus necessary to extend regularization methods, that can handle the situation where the number of covariates p largely exceed the sample size n, to the case in which covariates are also mismeasured. A variety of methods are available in this context, but many of them rely on knowledge about the measurement error and the structure of its covariance matrix. In this paper we set the goal to compare some of these methods, focusing on situations relevant for practical applications. In particular, we will evaluate these methods in setups in which the measurement error distribution and dependence structure are not known and have to be estimated from data. Our focus is on variable selection, and the evaluation is based on extensive simulations.

Screening methods for linear errors-in-variables models in high dimensions

“…We followed the procedures described in Sørensen et al (2015), Nghiem and Potgieter (2019) and Romeo and Thoresen (2019) to process the raw data using the BGX package of Hein et al (2005), and assumed the measurement error on each gene was mutually independent from that on the other. As a result, the measurement error covariance matrix Σ u was set to be diagonal.…”

“…Another approach for handling measurement error is to modify the loss function or the conditional score functions commonly seen in the error-free penalized regressions; examples include the corrected lasso method of Loh and Wainwright (2012) and Sørensen et al (2015), and the convex conditioned lasso of Datta et al (2017). More recently, Romeo and Thoresen (2019) presented a simulation study to compare the performance of the MU, corrected lasso, and convex conditioned lasso against the naive estimator in both estimation and variable selection; they concluded that the relative performance of those estimators depend on the structure of Σ u . Brown et al (2019) introduced a boosting algorithm based on the estimating equation of the corrected lasso, but the theoretical properties of the final estimates were not examined.…”

Screening Methods for Linear Errors-in-Variables Models in High Dimensions

“…Furthermore, a distinctive feature of the Affymetrix microarray data set is that multiple probes are used to measure each gene expression; hence, these replicated measurements can be used to estimate the covariance matrix of measurement errors 𝚺 𝑢 . We followed the procedures described in Sørensen et al (2015), Nghiem and Potgieter (2019), and Romeo and Thoresen (2019) to process the raw data using the BGX package of Hein et al (2005), and assumed the measurement error on each gene was mutually independent from that on the other. As a result, the measurement error covariance matrix 𝚺 𝑢 was set to be diagonal.…”

“…We followed the procedures described in Sørensen et al. (2015), Nghiem and Potgieter (2019), and Romeo and Thoresen (2019) to process the raw data using the BGX package of Hein et al. (2005), and assumed the measurement error on each gene was mutually independent from that on the other.…”

“…Zheng et al (2018) introduced a balanced estimator by adding nonconvex penalties to the objective function of the CoCo lasso estimator. More recently, Romeo and Thoresen (2019) presented a simulation study to compare the performance of the MU, corrected lasso, and CoCO lasso against the naive estimator in both estimation and variable selection; they concluded that the relative performance of those estimators depends on the structure of 𝚺 𝑢 . Brown et al (2019) introduced a boosting algorithm based on the estimating equation of the corrected lasso, but the theoretical properties of the final estimates were not examined.…”