Robust variable selection for nonlinear models with diverging number of parameters

Lv, Zhike; Zhu, Huiming; Yu, Keming

doi:10.1016/j.spl.2014.04.013

Cited by 9 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For global ap-proaches, the conditional mode is usually sought by maximizing the kernel density estimator for the variable induced by the residual and assuming that the global mode is unique and belongs to a certain hypothesis space. To name a few, studies in Lee (1989Lee ( , 1993; Lee and Kim (1998); Yao and Li (2014); Kemp and Santos Silva (2012); Baldauf and Santos Silva (2012); Yu and Aristodemou (2012); Lv et al (2014); Salah and Françoise (2016) follow this line. It should be noticed that most studies based upon global approaches assume the existence (and also the uniqueness) of a global conditional mode function that is of a parametric form.…”

Section: Historical Notes On Modal Regressionmentioning

confidence: 96%

A Statistical Learning Approach to Modal Regression

Feng

Fan

Suykens

2017

Preprint

View full text Add to dashboard Cite

This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially modal regression. We show that nonparametric modal regression problem can be approached via the classical empirical risk minimization. Some efforts are then made to develop a framework for analyzing and implementing modal regression. For instance, the modal regression function is described, the modal regression risk is defined explicitly and its Bayes rule is characterized; for the sake of computational tractability, the surrogate modal regression risk, which is termed as the generalization risk in our study, is introduced. On the theoretical side, the excess modal regression risk, the excess generalization risk, the function estimation error, and the relations among the above three quantities are studied rigorously. It turns out that under mild conditions, function estimation consistency and convergence may be pursued in modal regression as in vanilla regression protocols such as mean regression, median regression, and quantile regression. However, it outperforms these regression models in terms of robustness as shown in our study from a re-descending M-estimation view. This coincides with and in return explains the merits of MCCR on robustness. On the practical side, the implementation issues of modal regression including the computational algorithm and the tuning parameters selection are discussed. Numerical assessments on modal regression are also conducted to verify our findings empirically.

show abstract

Section: Historical Notes On Modal Regressionmentioning

confidence: 96%

A Statistical Learning Approach to Modal Regression

Feng

Fan

Suykens

2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Other related literatures include Yang et al(2014), Lv et al(2014), Zhu et al(2015) and so on. Since the modal regression not only has very good robustness in the presence of outliers or heavy-tail error distributions, but also achieves full asymptotic efficiency under the normal error distribution.…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Variable selection in partially linear additive models for modal regression

Yang

Guo

2017

Communications in Statistics - Simulation and Computation

View full text Add to dashboard Cite

Based on B-spline basis functions and smoothly clipped absolute deviation (SCAD) penalty, we present a new estimation and variable selection procedure based on modal regression for partially linear additive models. The outstanding merit of the new method is that it is robust against outliers or heavy-tail error distributions and performs no worse than the least-squaresbased estimation for normal error case. The main difference is that the standard quadratic loss is replaced by a kernel function depending on a bandwidth that can be automatically selected based on the observed data. With appropriate selection of the regularization parameters, the new method possesses the consistency in variable selection and oracle property in estimation.Finally, both simulation study and real data analysis are performed to examine the performance of our approach. B-spline; Modal regression; Robust estimation; Oracle property; Partially linear additive models.

show abstract

“…Shahriari and Ahmadi (2015) used S-estimator robust clustering of high dimensional data. Lv et al (2014) simultaneously selected variables and estimated nonlinear models based on modal regression (MR), when the number of coefficients diverges with sample size.…”

Section: Recent Developments and Future Directionsmentioning

confidence: 99%

Robustness, Data Analysis, and Statistical Modeling: The First 50 Years and Beyond

Barrios

2015

CSAM

View full text Add to dashboard Cite

We present a survey of contributions that defined the nature and extent of robust statistics for the last 50 years. From the pioneering work of Tukey, Huber, and Hampel that focused on robust location parameter estimation, we presented various generalizations of these estimation procedures that cover a wide variety of models and data analysis methods. Among these extensions, we present linear models, clustered and dependent observations, times series data, binary and discrete data, models for spatial data, nonparametric methods, and forward search methods for outliers. We also present the current interest in robust statistics and conclude with suggestions on the possible future direction of this area for statistical science.

show abstract

Robust variable selection for nonlinear models with diverging number of parameters

Cited by 9 publications

References 18 publications

A Statistical Learning Approach to Modal Regression

A Statistical Learning Approach to Modal Regression

Variable selection in partially linear additive models for modal regression

Robustness, Data Analysis, and Statistical Modeling: The First 50 Years and Beyond

Contact Info

Product

Resources

About