On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding

“…The LASSO has 17 the smallest PE but the largest MS for all cases. The SCAD and MCP perform better in terms of PE but have larger MS as a 18 increases, while the SCAD has larger MS than the MCP for the same a. Among 5 MCLs, the MCL with γ =γ selects much 19 smaller variables, even though it has slightly larger PE than the LASSO.…”

mentioning

confidence: 93%

Moderately clipped LASSO

Kwon

Lee

Kim

2015

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

“…But a recent paper by Kyung et al (2009) shows that if a true variable is zero then bootstrap estimation of that variable is not consistent. More broadly, only calculating the standard errors may be inadequate for describing the true uncertainty of the LASSO estimator since Pötscher and Leeb (2007) show the distribution of the LASSO estimator is a mixture of two truncated normal densities. Of course, it still gives a general sense of the variability of the estimator; perhaps just as important, we will see that it can be used to differentiate the good estimators from the bad ones in most cases.…”

Section: Evaluation Of the Fitmentioning

confidence: 99%

“…Finally we can resize vec(P A ) and vec(P A ) to M × N matrices and plot them as images. We should note that this is a point-wise standard deviation-simultaneous standard deviation surfaces are difficult to obtain since the distribution of u is complex (Pötscher and Leeb 2007).…”

Section: Evaluation Of the Fitmentioning

confidence: 99%

Functional Concurrent Linear Regression Model for Spatial Images

Zhang

Clayton

Townsend

2010

JABES

View full text Add to dashboard Cite

Motivated by a problem in describing forest nitrogen cycling, in this paper we explore regression models for spatial images. Specifically, we present a functional concurrent linear model with varying coefficients for two-dimensional spatial images. To address overparameterization issues, the parameter surfaces in this model are transformed into the wavelet domain and a sparse representation is found by using a large-scale l 1 constrained least squares algorithm. Once the sparse representation is identified, an inverse wavelet transform is applied to obtain the estimated parameter surfaces. The optimal penalization term in the objective function is determined using the Bayesian Information Criterion (BIC) and we introduce measures of model quality. Our model is versatile and can be applied to both single and multiple replicate cases.

show abstract

“…However, as proved by Zhao and Yu (2006), the crude lasso is not model-consistent unless some cumbersome conditions on the design matrix. Moreover, Zou and Hastie (2005) showed that it can be sensitive to highly correlated predictors and Pötscher and Leeb (2009) warned that its distributional properties can be surprisingly complex. A large number of proposals have been made to enhance the lasso as a selection operator.…”

Section: Penalized Likelihoodmentioning

confidence: 99%

Combining a relaxed EM algorithm with Occam’s razor for Bayesian variable selection in high-dimensional regression

Latouche

Mattei

Bouveyron

et al. 2016

Journal of Multivariate Analysis

View full text Add to dashboard Cite

Please cite this article as: P. Latouche, P.-A. Mattei, C. Bouveyron, J. Chiquet, Combining a relaxed EM algorithm with Occam's razor for Bayesian variable selection in high-dimensional regression, Journal of Multivariate Analysis (2015), http://dx. AbstractWe address the problem of Bayesian variable selection for high-dimensional linear regression. We consider a generative model that uses a spike-and-slab-like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. The originality of the work is to consider inference through relaxing the model and using a type-II log-likelihood maximization based on an EM algorithm. Model selection is performed afterwards relying on Occam's razor and on a path of models found by the EM algorithm. Numerical comparisons between our method, called spinyReg, and state-of-the-art high-dimensional variable selection algorithms (such as lasso, adaptive lasso, stability selection or spike-and-slab procedures) are reported. Competitive variable selection results and predictive performances are achieved on both simulated and real benchmark data sets. An original regression data set involving the prediction of the number of visitors of the Orsay museum in Paris using bike-sharing system data is also introduced, illustrating the efficiency of the proposed approach. The R package spinyReg implementing the method proposed in this paper is available on CRAN.

show abstract

On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding

Cited by 23 publications

References 20 publications

Moderately clipped LASSO

Moderately clipped LASSO

Functional Concurrent Linear Regression Model for Spatial Images

Combining a relaxed EM algorithm with Occam’s razor for Bayesian variable selection in high-dimensional regression

Contact Info

Product

Resources

About