Influence functions for penalized M-estimators

Avella‐Medina, Marco

doi:10.3150/16-bej841

Cited by 22 publications

(18 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…First, there are no general results regarding the level influence function of tests for these settings. Second, the influence function of nonparametric and high-dimensional penalized estimators has been formulated for a fixed tuning parameter (Christmann and Steinwart, 2007;Avella-Medina, 2017). Since in practice this parameter is usually chosen by some data driven criterion, it would be necessary to account for this selection step in the derivation of differentially private statistics following the approach of this work.…”

Section: Discussionmentioning

confidence: 99%

Privacy-Preserving Parametric Inference: A Case for Robust Statistics

Avella‐Medina

2020

Journal of the American Statistical Association

Self Cite

View full text Add to dashboard Cite

Differential privacy is a cryptographically-motivated approach to privacy that has become a very active field of research over the last decade in theoretical computer science and machine learning. In this paradigm one assumes there is a trusted curator who holds the data of individuals in a database and the goal of privacy is to simultaneously protect individual data while allowing the release of global characteristics of the database. In this setting we introduce a general framework for parametric inference with differential privacy guarantees. We first obtain differentially private estimators based on bounded influence M-estimators by leveraging their gross-error sensitivity in the calibration of a noise term added to them in order to ensure privacy. We then show how a similar construction can also be applied to construct differentially private test statistics analogous to the Wald, score and likelihood ratio tests. We provide statistical guarantees for all our proposals via an asymptotic analysis. An interesting consequence of our results is to further clarify the connection between differential privacy and robust statistics. In particular, we demonstrate that differential privacy is a weaker stability requirement than infinitesimal robustness, and show that robust M-estimators can be easily randomized in order to guarantee both differential privacy and robustness towards the presence of contaminated data. We illustrate our results both on simulated and real data.

show abstract

Section: Discussionmentioning

confidence: 99%

Privacy-Preserving Parametric Inference: A Case for Robust Statistics

Avella‐Medina

2020

Journal of the American Statistical Association

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, there is not much work on the theoretical characterization of robustness for these and more general methods. Some exceptions are [1,55], where the authors study the breakdown point of some penalized methods for linear models, [4], where a rigorous definition of the influence function of penalized M−estimators is provided, and [49], where the theoretical properties of an adaptive version of the Huber regression estimator is investigated.…”

Section: Robustifying Penalized Methodsmentioning

confidence: 99%

The main contributions of robust statistics to statistical science and a new challenge

Ronchetti

2020

METRON

View full text Add to dashboard Cite

In the first part of the paper, we trace the development of robust statistics through its main contributions which have penetrated mainstream statistics. The goal of this paper is neither to provide a full overview of robust statistics, nor to make a complete list of its tools and methods, but to focus on basic concepts that have become standard ideas and tools in modern statistics. In the second part we focus on the particular challenge provided by high-dimensional statistics and discuss how robustness ideas can be used and adapted to this situation. Keywords Data science • Statistical models • Neighborhoods of a model • Game theory • Minimax approach • Huber function • Statistical functionals • Gâteaux derivative • Fréchet derivative • Influence function • Breakdown point • M-estimators • Generalized Method of Moments • Generalized Estimating Equations • High-dimensional statistics • Saturated models • Penalized methods • Oracle properties

show abstract

“…The derivation of the influence function of lasso‐type penalized estimators appears to be somehow more difficult. Some progress about this topic has been done by Wang et al and Oellerer et al Avella‐Medina provides a rigorous theoretical derivation by defining an influence function as the limit of a sequence of differentiable approximations and by showing that it can be viewed as a derivative in the sense of distribution theory…”

Section: High‐dimensional Statisticsmentioning

confidence: 99%

Robust statistics: a selective overview and new directions

Medina

Ronchetti

2015

WIREs Computational Stats

View full text Add to dashboard Cite

Classical statistics relies largely on parametric models. Typically, assumptions are made on the structural and the stochastic parts of the model and optimal procedures are derived under these assumptions. Standard examples are least squares estimators in linear models and their extensions, maximum-likelihood estimators and the corresponding likelihood-based tests, and generalized methods of moments (GMM) techniques in econometrics. Robust statistics deals with deviations from the stochastic assumptions and their dangers for classical estimators and tests and develops statistical procedures that are still reliable and reasonably efficient in the presence of such deviations. It can be viewed as a statistical theory dealing with approximate parametric models by providing a reasonable compromise between the rigidity of a strict parametric approach and the potential difficulties of interpretation of a fully nonparametric analysis. Many classical procedures are well known for not being robust. These procedures are optimal when the assumed model holds exactly, but they are biased and/or inefficient when small deviations from the model are present. The statistical results obtained from standard classical procedures on real data applications can therefore be misleading. In this paper we will give a brief introduction to robust statistics by reviewing some basic general concepts and tools and by showing how they can be used in data analysis to provide an alternative complementary analysis with additional useful information. In this study, we focus on robust statistical procedures based on M-estimators and tests because they provide a unified statistical framework that complements the classical theory. Robust procedures will be discussed for standard models, including linear models, general linear model, and multivariate analysis. Some recent developments in high-dimensional statistics will also be outlined.

show abstract

Influence functions for penalized M-estimators

Cited by 22 publications

References 30 publications

Privacy-Preserving Parametric Inference: A Case for Robust Statistics

Privacy-Preserving Parametric Inference: A Case for Robust Statistics

The main contributions of robust statistics to statistical science and a new challenge

Robust statistics: a selective overview and new directions

Contact Info

Product

Resources

About