Privacy-Preserving Gaussian Process Regression – A Modular Approach to the Application of Homomorphic Encryption

Abstract

Much of machine learning relies on the use of large amounts of data to train models to make predictions. When this data comes from multiple sources, for example when evaluation of data against a machine learning model is offered as a service, there can be privacy issues and legal concerns over the sharing of data. Fully homomorphic encryption (FHE) allows data to be computed on whilst encrypted, which can provide a solution to the problem of data privacy. However, FHE is both slow and restrictive, so existing algorithms must be manipulated to make them work efficiently under the FHE paradigm. Some commonly used machine learning algorithms, such as Gaussian process regression, are poorly suited to FHE and cannot be manipulated to work both efficiently and accurately. In this paper, we show that a modular approach, which applies FHE to only the sensitive steps of a workflow that need protection, allows one party to make predictions on their data using a Gaussian process regression model built from another party's data, without either party gaining access to the other's data, in a way which is both accurate and efficient. This construction is, to our knowledge, the first example of an effectively encrypted Gaussian process.

DPSNN: A Differentially Private Spiking Neural Network

“…HE allows models to use encrypted data as input and ouput encrypted results, then the clients can decrypt them and get the results they want without data leakage. Many basic machine learning algorithms can be implemented with HE (Du et al, 2017;Bost et al, 2014;Fenner & Pyzer-Knapp, 2020). In some cases, data from different enterprises must be used to obtain a better model, but no one wants to leak their data.…”

A Secure Federated Data-Driven Evolutionary Multi-objective Optimization Algorithm

“…Finally, HE has been introduced in constructing the GP [25] for protecting data privacy. HE requires that the ciphertext can be directly operated algebraically (typically by addition or multiplication), and the result must be the same as that obtained by using plaintext operations followed by encryption.…”

“…HE is also studied along the line of BO to handle expensive problems, where computational cost is a key consideration. To reduce the computational cost, a modular approach to GP modeling is suggested [25]. As an extension of privacy-preserving GP regression to BO, the GP must be updated iteratively, making it computationally prohibitive.…”

A survey on security and privacy threats to federated learning

“…being exposed to the cloud server. A lot of work [91,79,66,37,174,125,168,100,142,169,34,163,97,271,164] emerged in order to address data leakage in the inference phase, and we will describe these studies in detail afterwards.…”

“…By using polynomials to approximate nonlinear activation functions, such as ReLU, Sigmoid, etc., they achieve the expected prediction performance of 99% on MNIST. Subsequently, to address the problem that HE is difficult to achieve secure computation of nonlinear operations, Fenner and Pyzer-Knapp [66] proposes a way of server-user interaction. Specifically, the server first completes the linear computation of SFI in the ciphertext state and sends the computation result to the user.…”