Nonparametrically consistent depth-based classifiers

Abstract

We introduce a class of depth-based classification procedures that are of a nearest-neighbor nature. Depth, after symmetrization, indeed provides the center-outward ordering that is necessary and sufficient to define nearest neighbors. Like all their depth-based competitors, the resulting classifiers are affine-invariant, hence in particular are insensitive to unit changes. Unlike the former, however, the latter achieve Bayes consistency under virtually any absolutely continuous distributions - a concept we call nonparametric consistency, to stress the difference with the stronger universal consistency of the standard $k$NN classifiers. We investigate the finite-sample performances of the proposed classifiers through simulations and show that they outperform affine-invariant nearest-neighbor classifiers obtained through an obvious standardization construction. We illustrate the practical value of our classifiers on two real data examples. Finally, we shortly discuss the possible uses of our depth-based neighbors in other inference problems.Comment: Published at http://dx.doi.org/10.3150/13-BEJ561 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

From Depth to Local Depth: A Focus on Centrality

“…If one assumes that (Q2) also holds for θ θ θ / ∈ Supp(f ), then it is easy to check that the proof of lemma A.1(i) in Paindaveine and Van Bever (2012) further extends to the case where the symmetry center does not belong to Supp(f ). Therefore, there still exist δ > 0 and α < α * x such that B x (δ) ⊂ R x,α ⊂ B x (ε).…”

“…The depth regions R α (P ) or R β (P ) provide neighborhoods of the deepest point(s) only, hence cannot be used for that purpose. However, in view of (P2)-(P3) in Definition 3.3, neighborhoods of any x ∈ R d can be obtained [as in Paindaveine and Van Bever (2012)] by replacing P = P X with its symmetrized version P x = 1 2 P X + 1 2 P 2x−X . In line with most depths, the resulting (depth-based) neighborhoods R α (P x ) or R β (P x ) are of a nonparametric nature.…”

“…Depth methods allow several inference problems to be addressed, including, for example, testing for location and scale differences based on the DD-plot (first introduced as a graphical display for data exploration; Liu, Parelius, and Singh 1999;Li and Liu 2004), diagnostics of nonnormality (Liu, Parelius, and Singh 1999), outlier detection (Chen et al 2009), etc. More recently, depth was used extensively in a classification context (see, among many others, Ghosh and Chaudhuri 2005;Dutta and Ghosh 2012;Li, Cuesta-Albertos, and Liu 2012;Paindaveine and Van Bever 2012).…”

“…The proposed local depth is defined as global depth conditional on some neighborhood of the point of interest. To make this local concept purely based on depth, we use the neighborhoods that were recently introduced (for classification purposes) in Paindaveine and Van Bever (2012). As we will show, the resulting local depths allow for interesting inferential applications.…”

“…In Section 3, we first review the basics of depth (Section 3.1). We then describe the depth-based neighborhoods from Paindaveine and Van Bever (2012) and show how they allow to define local depth (Section 3.2). We also establish consistency of the corresponding sample local depth (Section 3.3).…”