Strongly universally consistent nonparametric regression and classification with privatised data

Abstract

In this paper we revisit the classical problem of nonparametric regression, but impose local differential privacy constraints. Under such constraints, the raw data (X 1 , Y 1 ), . . . , (Xn, Yn), taking values in R d × R, cannot be directly observed, and all estimators are functions of the randomised output from a suitable privacy mechanism. The statistician is free to choose the form of the privacy mechanism, and here we add Laplace distributed noise to a discretisation of the location of a feature vector X i and to the value of its response variable Y i . Based on this randomised data, we design a novel estimator of the regression function, which can be viewed as a privatised version of the well-studied partitioning regression estimator. The main result is that the estimator is strongly universally consistent, and we further establish an upper bound on the rate of convergence. Our methods and analysis also give rise to a strongly universally consistent binary classification rule for locally differentially private data.

Locally private online change point detection

“…where {ǫ i,j , ζ i,j } are independent and identically distributed standard Laplace random variables, and where [Y ] M −M = min(M, max(Y, −M )) with M > 0 a truncation parameter. It is shown in Proposition 1 in Berrett et al (2021) (see also Berrett and Butucea, 2019) that this non-interactive mechanism is an α-LDP channel.…”

“…The construction of estimators is detailed in Definition 2. This estimator has previously studied in the one sample estimation scenario in Berrett et al (2021), where it was shown to be a universally strongly consistent estimator of the true regression function.…”

“…Yu et al, 2021). On a separate note, multivariate nonparametric regression estimation, under privacy constraints, is studied in Berrett et al (2021), and classification is studied in Berrett and Butucea (2019) In addition, we have also provided the analysis and results based on the univariate mean online change point detection problem, with privatised data. This is, arguably, the simplest privatised, online change point detection problem.…”

On rate optimal private regression under local differential privacy

“…More precisely, the anonymised data must satisfy a local differential privacy (LDP) condition. Our work is motivated by the recent paper (Berrett, Györfi, and Walk, 2021) where a first step in this direction was done. In that paper the authors considered a private partitioning estimate and derived the upper bound n −1/(d+1) on the rate of convergence for Lipschitz continuous functions (β = 1).…”

“…In that paper the authors considered a private partitioning estimate and derived the upper bound n −1/(d+1) on the rate of convergence for Lipschitz continuous functions (β = 1). However, the rate was only established under quite a restrictive assumption (called strong density assumption (SDA) in (Berrett, Györfi, and Walk, 2021)) on the design distribution µ. Moreover, it was conjectured that the rate of convergence can be arbitrarily slow when the SDA is not fulfilled.…”