On the robustness to adversarial corruption and to heavy-tailed data of the Stahel–Donoho median of means

Abstract

We consider median of means (MOM) versions of the Stahel–Donoho outlyingness (SDO) [ 23, 66] and of the Median Absolute Deviation (MAD) [ 30] functions to construct subgaussian estimators of a mean vector under adversarial contamination and heavy-tailed data. We develop a single analysis of the MOM version of the SDO which covers all cases ranging from the Gaussian case to the $L_2$ case. It is based on isomorphic and almost isometric properties of the MOM versions of SDO and MAD. This analysis also covers cases where the mean does not even exist but a location parameter does; in those cases we still recover the same subgaussian rates and the same price for adversarial contamination even though there is not even a first moment. These properties are achieved by the classical SDO median and are therefore the first non-asymptotic statistical bounds on the Stahel–Donoho median complementing the $\sqrt{n}$-consistency [ 58] and asymptotic normality [ 74] of the Stahel–Donoho estimators. We also show that the MOM version of MAD can be used to construct an estimator of the covariance matrix only under the existence of a second moment or of a scatter matrix if a second moment does not exist.

Statistically Optimal Robust Mean and Covariance Estimation for Anisotropic Gaussians

“…That is, one aims to construct an estimator µ such that ( µ−µ) T Σ −1 ( µ−µ) is small with high probability. It appears that the bounds with respect to this norm are necessarily dimension-dependent, and a simple VC-type/sphere covering argument is sufficient to obtain the optimal rates of convergence [DL22]. Similar observations are also valid for the covariance estimation problem.…”

“…Another interesting aspect of our analysis is that we do not make any assumptions on the sample size. In comparison, the existing estimators that recover the optimal bound in the isotropic case [CGR18,DK19] require N c(d + log(1/δ)), or N cdε −2 as in [DL22], where c > 0 is some absolute constant. We will encounter some weaker assumptions in Section 6, but only when tuning a single real-valued parameter of our estimator.…”

“…It is worth mentioning that there are several results showing that the linear dependence on ε can be achieved in the isotropic case, where Σ is identity (or close to it). We refer to the analysis of the Tukey median, the direction-dependent median as well as the Stahel-Donoho median of means in respectively [CGR18,DK19,DL22]. Apart from the fact that the isotropic assumption is quite restrictive when working with real data, the presence of VC-type or sphere covering arguments is inherent to the analysis of existing dimension-dependent estimators.…”

“…whenever N c log(1/δ) and ε is smaller than some absolute constant. When going to higher dimensions, instead of working with Tukey's median, whose sharp analysis is only known in the isotropic case [CGR18], or Stahel-Donoho-type estimators as in [DL22], we base our solution on what we call the smoothed median estimator.…”

Sharper Concentration Inequalities for Median-of-Mean Processes

“…[9] studied the non-asymptotic concentration of the heteroskedastic Wishart-type matrices; Ref. [10] constructed sub-Gaussian estimators of a mean vector under adversarial contamination and heavy-tailed data by Median-of-Mean versions of the Stahel-Donoho outlyingness and of Median Absolute Deviation functions; Ref. [11] obtained the deconvolution for some singular density errors via a combinatorial Median-of-Mean approach and assessed the estimator quality by establishing non-asymptotic risk bounds.…”