Deep Neural Networks Classification over Encrypted Data

Abstract

Machine learning algorithms based on deep neural networks have achieved remarkable results and are being extensively used in different domains. However, the machine learning algorithms requires access to raw data which is often privacy sensitive. To address this issue, we develop new techniques to provide solutions for running deep neural networks over encrypted data. In this paper, we develop new techniques to adopt deep neural networks within the practical limitation of current homomorphic encryption schemes. More specifically, we focus on classification of the well-known convolutional neural networks (CNN). First, we design methods for approximation of the activation functions commonly used in CNNs (i.e. ReLU, Sigmoid, and Tanh) with low degree polynomials which is essential for efficient homomorphic encryption schemes. Then, we train convolutional neural networks with the approximation polynomials instead of original activation functions and analyze the performance of the models. Finally, we implement convolutional neural networks over encrypted data and measure performance of the models. Our experimental results validate the soundness of our approach with several convolutional neural networks with varying number of layers and structures. When applied to the MNIST optical character recognition tasks, our approach achieves 99.52% accuracy which significantly outperforms the state-of-the-art solutions and is very close to the accuracy of the best non-private version, 99.77%. Also, it can make close to 164000 predictions per hour. We also applied our approach to CIFAR-10, which is much more complex compared to MNIST, and were able to achieve 91.5% accuracy with approximation polynomials used as activation functions. These results show that CryptoDL provides efficient, accurate and scalable privacy-preserving predictions. *

Applying Deep Neural Networks over Homomorphic Encrypted Medical Data

“…As ciphertext data are centralized to one single entity, it does not imply any communication bottleneck, as compared to SMPC. Although early work on HE [23] involves highly intensive computations, making the method infeasible for the machine learning algorithm, the recent subsequent schemes gave rise to a series of privacy-preserving machine learning solutions [24][25][26][27][28][29].…”

AutoGAN-based dimension reduction for privacy preservation

“…Thus, such additive homomorphic encryption-based methods in [1,2] are only applicable to simple machine learning algorithms such as decision tree and naive bayes. In Hesamifard's work [3],the fully homomorphic encryption is applied to perform deep neural networks over encrypted data, where the non-linear activation functions are approximated by polynomials. In secure multi-party computing (SMC), multiple parties collaborate to compute functions without revealing plain-text to other parties.…”

“…Several tools and methods have been developed to preserve the privacy in machine learning, such as homomorphic encryption [1][2][3], secure multi-party computing [4,5], differential privacy (DP) [6][7][8][9][10], compressive privacy [11][12][13][14][15]. Differential privacy-based methods aim at preventing leaking individual information caused by queries.…”

CryptoNN: Training Neural Networks over Encrypted Data

“…To tackle the challenge of training a model over encrypted data in a simpler manner, in this paper, we propose a novel [3], [9], [10], [11], [12], [13], [14], [15] • • Covers All Homomorphic Encryption (HE) [2] •…”