Differentially Private Algorithms for Learning Mixtures of Separated Gaussians

Kamath, Gautam; Sheffet, Or; Singhal, Vikrant; Ullman, Jonathan

doi:10.1109/ita50056.2020.9244945

Cited by 31 publications

(49 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By now, there are various private algorithms that learn the parameters of a single Gaussian [KV18; KLSU19; CWZ19; BS19; KSU20; BDKU20]. Recently, [KSSU19] presented a private algorithm for learning mixtures of well-separated (and bounded) Gaussians. We remark, however, that besides the result of [BDKU20], which is a practical algorithm for learning a single Gaussian, all the other results are primarily theoretical.…”

Section: K-means Clusteringmentioning

confidence: 99%

“…By a reduction to the k-tuples clustering problem, we present a simple algorithm that privately learns the parameters of a separated (and bounded) mixture of k Gaussians. From a practical perspective, compared with the construction of the main algorithm of [KSSU19], our algorithm is simple and implementable. From a theoretical perspective, our algorithm offers reduced sample complexity, weaker separation assumption, and modularity.…”

Section: K-means Clusteringmentioning

confidence: 99%

“…In the work of [KSSU19], they presented an alternative algorithm for learning mixtures of Gaussians, which optimizes the sample-and-aggregate approach of [NRS07], and is somewhat similar to our approach. That is, their algorithm executes a non-private algorithm several times, each time for obtaining a new "k-tuple" of means estimations, and then aggregates the findings by privately determine a new k-tuple of means estimation.…”

Section: Other Related Workmentioning

confidence: 99%

“…The main private algorithm of [KSSU19] mimics the approach of the (non-private) algorithm of [AM05], which is to use PCA to project the data into a low-dimensional space, and then clustering the data points in that low-dimensional space. This projection enable both algorithms to learn mixtures that have the following separation ∀i, j :…”

Section: Comparison To the Main Algorithm Of [Kssu19]mentioning

confidence: 99%

See 3 more Smart Citations

Differentially-Private Clustering of Easy Instances

Cohen¹,

Kaplan²,

Mansour³

et al. 2021

Preprint

View full text Add to dashboard Cite

Clustering is a fundamental problem in data analysis. In differentially private clustering, the goal is to identify k cluster centers without disclosing information on individual data points. Despite significant research progress, the problem had so far resisted practical solutions. In this work we aim at providing simple implementable differentially private clustering algorithms that provide utility when the data is "easy," e.g., when there exists a significant separation between the clusters.We propose a framework that allows us to apply non-private clustering algorithms to the easy instances and privately combine the results. We are able to get improved sample complexity bounds in some cases of Gaussian mixtures and k-means. We complement our theoretical analysis with an empirical evaluation on synthetic data.

show abstract

Section: K-means Clusteringmentioning

confidence: 99%

Section: K-means Clusteringmentioning

confidence: 99%

Section: Other Related Workmentioning

confidence: 99%

Section: Comparison To the Main Algorithm Of [Kssu19]mentioning

confidence: 99%

See 2 more Smart Citations

Differentially-Private Clustering of Easy Instances

Cohen¹,

Kaplan²,

Mansour³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this work, we use simple non-hybrid baseline mean estimators to enable us to obtain exact finite-sample utility expressions. However, DP mean estimation of distributions under both the TCM and LM has been studied since the models' introductions [17,21,56], and continues to be actively studied to this day [2,15,18,20,22,28,29,32,37,[39][40][41][42]. The goal of mean estimation research under both models is to maximize utility while minimizing the sample complexity by making various distributional assumptions.…”

Section: Non-hybrid Mean Estimationmentioning

confidence: 99%

The Power of the Hybrid Model for Mean Estimation

Avent

Dubey

Korolova

2020

Proceedings on Privacy Enhancing Technologies

View full text Add to dashboard Cite

We explore the power of the hybrid model of differential privacy (DP), in which some users desire the guarantees of the local model of DP and others are content with receiving the trusted-curator model guarantees. In particular, we study the utility of hybrid model estimators that compute the mean of arbitrary realvalued distributions with bounded support. When the curator knows the distribution’s variance, we design a hybrid estimator that, for realistic datasets and parameter settings, achieves a constant factor improvement over natural baselines.We then analytically characterize how the estimator’s utility is parameterized by the problem setting and parameter choices. When the distribution’s variance is unknown, we design a heuristic hybrid estimator and analyze how it compares to the baselines. We find that it often performs better than the baselines, and sometimes almost as well as the known-variance estimator. We then answer the question of how our estimator’s utility is affected when users’ data are not drawn from the same distribution, but rather from distributions dependent on their trust model preference. Concretely, we examine the implications of the two groups’ distributions diverging and show that in some cases, our estimators maintain fairly high utility. We then demonstrate how our hybrid estimator can be incorporated as a sub-component in more complex, higher-dimensional applications. Finally, we propose a new privacy amplification notion for the hybrid model that emerges due to interaction between the groups, and derive corresponding amplification results for our hybrid estimators.

show abstract