Fast Homomorphic Evaluation of Deep Discretized Neural Networks

Abstract

The rise of machine learning as a service multiplies scenarios where one faces a privacy dilemma: either sensitive user data must be revealed to the entity that evaluates the cognitive model (e.g., in the Cloud), or the model itself must be revealed to the user so that the evaluation can take place locally. Fully Homomorphic Encryption (FHE) offers an elegant way to reconcile these conflicting interests in the Cloudbased scenario and also preserve non-interactivity. However, due to the inefficiency of existing FHE schemes, most applications prefer to use Somewhat Homomorphic Encryption (SHE), where the complexity of the computation to be performed has to be known in advance, and the efficiency of the scheme depends on this global complexity.In this paper, we present a new framework for homomorphic evaluation of neural networks, that we call FHE-DiNN, whose complexity is strictly linear in the depth of the network and whose parameters can be set beforehand. To obtain this scale-invariance property, we rely heavily on the bootstrapping procedure. We refine the recent FHE construction by Chillotti et al. (ASIACRYPT 2016) in order to increase the message space and apply the sign function (that we use to activate the neurons in the network) during the bootstrapping. We derive some empirical results, using TFHE library as a starting point, and classify encrypted images from the MNIST dataset with more than 96% accuracy in less than 1.7 seconds.Finally, as a side contribution, we analyze and introduce some variations to the bootstrapping technique of Chillotti et al. that offer an improvement in efficiency at the cost of increasing the storage requirements.

Survey on Federated Learning Towards Privacy Preserving AI

“…The authors of [33] used homomorphic encryption to enhance the security of the entire system and preserve the privacy of gradients. With recent trends in machine learning and AI, privacy-preserving neural networks are also one of the research interest [34,35,36,37]. Therefore, building a decentralized system with collaborative machine learning models and ensuring data privacy is one of the crucial aspects of many industries.…”

MOBIUS: Model-Oblivious Binarized Neural Networks

“…Moreover, encrypted training is the out scope of this work. As related works on combining BNNs with cryptography, TAPAS [14] and FHE-DiNN [12] based on FHE have been concurrently proposed. FHE-DiNN utilized discretized neural networks where domains are defined from −w to +w, but its experiments were conducted with −1 to +1 exactly the same as BNNs.…”

“…The neural network is fully connected. We them compare the performance of our protocol with SecureML and other state-of-the-art protocols with cryptography [5], [12], [14].…”

Secure Architectures Implementing Trusted Coalitions for Blockchained Distributed Learning (TCLearn)

“…Solving those problems has been the subject of some recent work. As an example, in [25], Bourse et al presented a framework for homomorphic evaluation of neural networks using a highly optimized FHE algorithm.…”