PPEM: Privacy‐preserving EM learning for mixture models

Abstract

Summary Privacy is becoming increasingly important in collaborative data analysis, especially those involving personal or sensitive information commonly arising from health and commercial settings. The aim of privacy preserving statistical algorithms is to allow inference to be drawn on the joint data without disclosing private data held by each party. This paper presents a privacy‐preserving expectation–maximization (PPEM) algorithm for carrying out maximum likelihood estimation of the parameters of mixture models. We address the scenario of horizontally partitioned data distributed among three or more parties. The PPEM algorithm is a two‐cycle iterative distributed algorithm for fitting mixture models under privacy‐preserving requirements. A distinct advantage of PPEM is that it does not require a trusted third party for cooperative learning, unlike most existing schemes that implement a master/slave hierarchy. By adopting a ring topology and adding random noises to messages before encryption, PPEM helps prevent information leakage in the case of corrupted parties. Furthermore, in contrast to existing works, which typically assume a Honest‐but‐Curious adversary, we consider the much stronger case of a Malicious adversary. For illustration, PPEM is applied to two of the most popular mixture models, namely, the normal mixture model (NMM) and t‐mixture model (tMM), and their effectiveness is analyzed through a security analysis. A real data example is also presented to evaluate the computational complexity and accuracy of PPEM relative to its non‐privacy‐preserving version.

A systematic review on privacy-preserving distributed data mining

“…We decided to choose the latter approach, as we deemed our search criteria to be broad enough to cover follow-up studies. We have found several follow-up papers, where these papers present an extension of their existing methods to: 1) solve other data partitioning problems [125,126]; 2) apply to more advanced data analysis algorithms [62,64]; 3) to include more complicated user scenarios [67,68]; 4) to conduct more experiments by using real-life datasets [25,59,96,109].…”

Mixture-Model-Based Graph for Privacy-Preserving Semi-Supervised Learning

“…Reference [39] introduces a circular transition protocol for secure summation, which is only secure with an honest-majority. Utilizing additive homomorphic encryption schemes, references [40]- [42] each proposed a cryptographybased secure summation protocol against a malicious majority. In [39], [40], [42] a master site is randomly selected to obtain the results of secure summations for A i , B i , C i , D i , n, update the parameters according to (12)(13)(15) and distribute them to all other sites.…”

“…Utilizing additive homomorphic encryption schemes, references [40]- [42] each proposed a cryptographybased secure summation protocol against a malicious majority. In [39], [40], [42] a master site is randomly selected to obtain the results of secure summations for A i , B i , C i , D i , n, update the parameters according to (12)(13)(15) and distribute them to all other sites. Reference [41] proposed a protocol where every site initiates a secure summation as a sponsor and receives…”

Privacy-Preserving Distributed Expectation Maximization for Gaussian Mixture Model Using Subspace Perturbation

“…There is a substantial amount of work dealing with privacy in machine learning methods, see for instance [7][8][9][10][11][12][13][14]. Of particular interest to our work is [15][16][17][18][19][20][21] that also considers privacy of distributed EM. Among them, the privacy of the proposed solution in [21] is based on two party cryptographic computations that tend to be computationally heavy and time consuming.…”