Regularization in statistics

Bickel, Peter J.; Li, Bo; Tsybakov, Alexandre B.; Geer, Sara A. van de; Yu, Bin; Valdés, Teófilo; Rivero, Carlos; Fan, Jianqing; Vaart, Aad van der

doi:10.1007/bf02607055

Cited by 189 publications

(133 citation statements)

References 106 publications

Supporting

Mentioning

131

Contrasting

Unclassified

Order By: Relevance

“…There exists a vast literature for statistics that describes the importance of regularization, particularly when analyzing high-dimensional data with small sam-ples (see Bickel & Li, 2006, for a detailed overview). Regularization is a useful technique for producing more accurate estimates by the use of prior knowledge.…”

mentioning

confidence: 99%

Exploratory factor analysis for small samples

2011

View full text Add to dashboard Cite

Traditionally, two distinct approaches have been employed for exploratory factor analysis: maximum likelihood factor analysis and principal component analysis. A third alternative, called regularized exploratory factor analysis, was introduced recently in the psychometric literature. Small sample size is an important issue that has received considerable discussion in the factor analysis literature. However, little is known about the differential performance of these three approaches to exploratory factor analysis in a small sample size scenario. A simulation study and an empirical example demonstrate that regularized exploratory factor analysis may be recommended over the two traditional approaches, particularly when sample sizes are small (below 50) and the sample covariance matrix is near singular.

show abstract

mentioning

confidence: 99%

Exploratory factor analysis for small samples

2011

View full text Add to dashboard Cite

show abstract

“…Golan thanks the Edwin T. Jaynes International Center for Bayesian Methods and Maximum Entropy for partially supporting this project. We also want to thank the referees for their comments, for their careful reading of the manuscript and for pointing out reference [26] to us. Their comments improved the presentation considerably.…”

Section: Acknowledgmentsmentioning

confidence: 98%

“…If, in addition, the data are ill-conditioned, one often has to resort to the class of regularization methods (e.g., Hoerl and Kennard [19] O'Sullivan [20], Breiman [21], Tibshirani [22], Titterington [23], Donoho et al [24]; Besnerais et al [25]. A reference for regularization in statistics is Bickel and Li [26]. If some prior information on the data generation process or on the model is available, Bayesian methods are often used.…”

Section: Introductionmentioning

confidence: 99%

An Entropic Estimator for Linear Inverse Problems

Golan

Gzyl

2012

Entropy

View full text Add to dashboard Cite

In this paper we examine an Information-Theoretic method for solving noisy linear inverse estimation problems which encompasses under a single framework a whole class of estimation methods. Under this framework, the prior information about the unknown parameters (when such information exists), and constraints on the parameters can be incorporated in the statement of the problem. The method builds on the basics of the maximum entropy principle and consists of transforming the original problem into an estimation of a probability density on an appropriate space naturally associated with the statement of the problem. This estimation method is generic in the sense that it provides a framework for analyzing non-normal models, it is easy to implement and is suitable for all types of inverse problems such as small and or ill-conditioned, noisy data. First order approximation, large sample properties and convergence in distribution are developed as well. Analytical examples, statistics for model comparisons and evaluations, that are inherent to this method, are discussed and complemented with explicit examples.

show abstract

“…This technique is particularly appealing for dealing with large-dimensional problems. A review article on regularization in statistics can be found in Bickel and Li (2006). This section overviews some regularized estimation methods.…”

Section: Regularized Estimation With Independent Datamentioning

confidence: 99%

Regularized estimation of hemodynamic response function for fMRI data

Zhang

2010

Statistics and Its Interface

View full text Add to dashboard Cite

One of the primary goals in analyzing fMRI data is to estimate the Hemodynamic Response Function (HRF), which is a large-dimensional parameter vector possessing some form of sparsity. This paper introduces a varyingdimensional model for the HRF, and develops novel regularization methods for estimating the HRF from fMRI time series via incorporating the sparsity feature. Particularly, we present three types of penalty choice methods: the Lasso, the adaptive Lasso and the SCAD. Simulation studies demonstrate the advantages of regularization methods, in terms of sparsity recovery, over conventional non-regularized approaches which restrict the HRF to be fixed low dimensional without capturing the sparsity structure. We illustrate the regularized methods for estimating the HRF using a real fMRI data set and compare with results offered by a popular imaging analysis tool AFNI.

show abstract

Regularization in statistics

Cited by 189 publications

References 106 publications

Exploratory factor analysis for small samples

Exploratory factor analysis for small samples

An Entropic Estimator for Linear Inverse Problems

Regularized estimation of hemodynamic response function for fMRI data

Contact Info

Product

Resources

About