CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU

Tan, Sijun; Knott, Brian; Tian, Yuan; Wu, David J.

doi:10.1109/sp40001.2021.00098

Cited by 96 publications

(70 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another approach is to replace GCs with secure enclaves and process linear layers on GPUs for more performance [19]. Some have also focused on privacy enhanced training [20,21], typically assuming a different threat model than this work.…”

Section: Resultsmentioning

confidence: 99%

Circa: Stochastic ReLUs for Private Deep Learning

Ghodsi,

Jha,

Reagen

et al. 2021

Preprint

View full text Add to dashboard Cite

The simultaneous rise of machine learning as a service and concerns over user privacy have increasingly motivated the need for private inference (PI). While recent work demonstrates PI is possible using cryptographic primitives, the computational overheads render it impractical. The community is largely unprepared to address these overheads, as the source of slowdown in PI stems from the ReLU operator whereas optimizations for plaintext inference focus on optimizing FLOPs. In this paper we re-think the ReLU computation and propose optimizations for PI tailored to properties of neural networks. Specifically, we reformulate ReLU as an approximate sign test and introduce a novel truncation method for the sign test that significantly reduces the cost per ReLU. These optimizations result in a specific type of stochastic ReLU. The key observation is that the stochastic fault behavior is well suited for the fault-tolerant properties of neural network inference. Thus, we provide significant savings without impacting accuracy. We collectively call the optimizations Circa and demonstrate improvements of up to 4.7× storage and 3× runtime over baseline implementations; we further show that Circa can be used on top of recent PI optimizations to obtain 1.8× additional speedup.Preprint. Under review.

show abstract

Section: Resultsmentioning

confidence: 99%

Circa: Stochastic ReLUs for Private Deep Learning

Ghodsi,

Jha,

Reagen

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Mohassel et al [22] propose the first secure training method for shallow neural networks under a two-server setting, based on secret sharing (for linear operations) and garbled circuits (for approximated activation functions). Subsequently, several works [23]- [26] achieved better performance in terms of accuracy and efficiency by devising customized secure training protocols in a three-server setting.…”

Section: Secure Neural Network Training and Inferencementioning

confidence: 99%

“…Similar to prior security designs in the three-server setting [24], [26], [29], we consider a semi-honest adversary setting where each of the three cloud servers honestly follow our protocol, but may individually attempt to learn the private information of the data owner. We consider that the data owner wishes to keep the following information private: (i) the features F and labels T of nodes, (ii) the adjacency matrix A encoding the structural information regarding the neighboring nodes of each node, the number of neighbors of each node, and the edge weight between each pair of connected nodes, (iii) the model weights M (1) and M (2) , and (iv) the inference results for (unlabeled) nodes.…”

Section: Threat Modelmentioning

confidence: 99%

“…We note that prior works (e.g., [23], [25], [26]) on secure CNN training generally terminate the training process at a specified number of epochs. However, the convergence of the training process is unpredictable, which can depend on various factors such as the training data set, the learning parameter setting, and random factors in the nature of model training.…”

Section: Secure Model Convergence Evaluationmentioning

confidence: 99%

“…A number of research efforts have been proposed for secure neural network inference and training. Most of existing works are focused on [13]- [21] designing specialized protocols for secure inference, and only a few works study [22]- [26] secure training which is more sophisticated and resource-intensive. However, prior works are all focused on the support for Convolutional Neural Networks (CNNs) handling unstructured data like images and text.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SecGNN: Privacy-Preserving Graph Neural Network Training and Inference as a Cloud Service

Wang¹,

Zheng²,

Jia³

2022

Preprint

View full text Add to dashboard Cite

Graphs are widely used to model the complex relationships among entities. As a powerful tool for graph analytics, graph neural networks (GNNs) have recently gained wide attention due to its end-to-end processing capabilities. With the proliferation of cloud computing, it is increasingly popular to deploy the services of complex and resource-intensive model training and inference in the cloud due to its prominent benefits. However, GNN training and inference services, if deployed in the cloud, will raise critical privacy concerns about the information-rich and proprietary graph data (and the resulting model). While there has been some work on secure neural network training and inference, they all focus on convolutional neural networks handling images and text rather than complex graph data with rich structural information. In this paper, we design, implement, and evaluate SecGNN, the first system supporting privacy-preserving GNN training and inference services in the cloud. SecGNN is built from a synergy of insights on lightweight cryptography and machine learning techniques. We deeply examine the procedure of GNN training and inference, and devise a series of corresponding secure customized protocols to support the holistic computation. Extensive experiments demonstrate that SecGNN achieves comparable plaintext training and inference accuracy, with practically affordable performance.

show abstract

Introduction

Ren

2022

Wireless Networks

View full text Add to dashboard Cite

CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU

Cited by 96 publications

References 28 publications

Circa: Stochastic ReLUs for Private Deep Learning

Circa: Stochastic ReLUs for Private Deep Learning

SecGNN: Privacy-Preserving Graph Neural Network Training and Inference as a Cloud Service

Introduction

Contact Info

Product

Resources

About