The Power of the Hybrid Model for Mean Estimation

Abstract

We explore the power of the hybrid model of differential privacy (DP), in which some users desire the guarantees of the local model of DP and others are content with receiving the trusted-curator model guarantees. In particular, we study the utility of hybrid model estimators that compute the mean of arbitrary realvalued distributions with bounded support. When the curator knows the distribution’s variance, we design a hybrid estimator that, for realistic datasets and parameter settings, achieves a constant factor improvement over natural baselines.We then analytically characterize how the estimator’s utility is parameterized by the problem setting and parameter choices. When the distribution’s variance is unknown, we design a heuristic hybrid estimator and analyze how it compares to the baselines. We find that it often performs better than the baselines, and sometimes almost as well as the known-variance estimator. We then answer the question of how our estimator’s utility is affected when users’ data are not drawn from the same distribution, but rather from distributions dependent on their trust model preference. Concretely, we examine the implications of the two groups’ distributions diverging and show that in some cases, our estimators maintain fairly high utility. We then demonstrate how our hybrid estimator can be incorporated as a sub-component in more complex, higher-dimensional applications. Finally, we propose a new privacy amplification notion for the hybrid model that emerges due to interaction between the groups, and derive corresponding amplification results for our hybrid estimators.

Advances and Open Problems in Federated Learning

“…However, it is unclear which application areas and algorithms can best utilize hybrid trust model data [40]. Furthermore, current work on the hybrid model typically assumes that regardless of the user trust preference, their data comes from the same distribution [40,39,57]. Relaxing this assumption is critical for FL in particular, as the relationship between the trust preference and actual user data may be non-trivial.…”

A Bias-Variance-Privacy Trilemma for Statistical Estimation

“…Several recent works explore connections between private and robust estimation [LKKO21, HKM22, GH22, LKO22, KMV22, AKT + 22, HKMN22, CCd + 23] and between privacy and generalization [HU14, DFH + 15, SU15, BNS + 16a, RRST16, FS17]. Emerging directions of interest include guaranteeing privacy when one person may contribute multiple samples [LSY + 20, LSA + 21, GRST22], a combination of local and central DP for different users [ADK19], or estimation with access to some public data [BKS22]. See [KU20] for more coverage of recent work on private statistical estimation.…”

New Lower Bounds for Private Estimation and a Generalized Fingerprinting Lemma

“…Other works study private statistical estimation in different and more general settings, including mixtures of Gaussians [KSSU19, AAL21], graphical models [ZKKW20], discrete distributions [DHS15], and median estimation [AMB19,TVGZ20]. Some recent directions involve guaranteeing user-level privacy [LSY `20, LSA `21], or a combination of local and central DP for different users [ADK19]. See [KU20] for further coverage of DP statistical estimation.…”