SecureBP from Homomorphic Encryption

Abstract

We present a secure backpropagation neural network training model (SecureBP), which allows a neural network to be trained while retaining the confidentiality of the training data, based on the homomorphic encryption scheme. We make two contributions. The first one is to introduce a method to find a more accurate and numerically stable polynomial approximation of functions in a certain interval. The second one is to find a strategy of refreshing ciphertext during training, which keeps the order of magnitude of noise at O˜e33.

PPNNBP: A Third Party Privacy-Preserving Neural Network With Back-Propagation Learning

“…In the context of NN there are two MLaaS variations; Training as a Service (TaaS) and Prediction as a Service (PaaS). In TaaS, a cloud provider sells services that allow data owners to upload their encrypted data and receive a trained NN model [18], [33], [35], [36], [38], [39]. For preserving data privacy, the trained model weights, biases and intermediate calculations as activation functions are encrypted.…”

“…Secure PaaS mechanisms that do not require recourse to clients, as in the case of [14], [30], typically work by using some form of alternative activation functions that can operate in encrypted form. A popular choice here is a quadratic activation function, f (x ) = x 2 , that can be evaluated using FHE multiplication properties as presented in [18], [37], [40]. However, the quadratic function results in accuracy loss [41].…”

“…However, the quadratic function results in accuracy loss [41]. Whatever the case, the replacement of the activation function does not address the substantial involvement of clients or data owners, in PaaS or TaaS, as FHE schemes typically required the execution of noise management techniques or required the re-encrypting of cyphertexts when noise exceeds a predetermined level [17], [18], [37]; this is clearly undesirable. To reduce the amount of time required to re-encrypt cyphertexts (the principal communication overhead), the level of the FHE scheme used can be increased [26].…”

“…This is typically resolved, in many FHE schemes, by some form of noise management techniques such as bootstrapping, modulus switching, scale invariant and flattering [11], [16]. However, these techniques tend to be complex [17], [18]. Recently, noise-free FHE schemes have been proposed whereby noise management can be avoided by allowing an arbitrary large quantity of noise [19], [20].…”

“…Similarly, usage of the trained NN is conducted in an encrypted manner (both input data and predicted output). The PPNNBP framework features only minimal data owner participation during training and usage, unlike alternative frameworks [14], [17], [18], [37], whereas achieving an accuracy comparable with that of identical networks trained without encryption.…”