Model Protection: Real-Time Privacy-Preserving Inference Service for Model Privacy at the Edge

Hou, Jiahui; Liu, Huiqi; Liu, Yunxin; Wang, Yu; Wan, Peng‐Jun; Li, Xiangyang

doi:10.1109/tdsc.2021.3126315

Cited by 21 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from the above works, we foucs on reducing the SLO violation rate caused by the interference of multimodel. In addition, some edge inference frameworks involve privacy protection [21], [22] and edge-cloud collaborative [23], [24], respectively. These works are orthogonal to BCEdge that can alleviate privacy and resource constraints.…”

Section: A Model-level Dnn Inference Servicementioning

confidence: 99%

BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms

Zhang¹,

Li²,

Zhao³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>As deep neural networks (DNNs) are being applied to a wide range of edge intelligent applications, it is critical for edge inference platforms to have both high-throughput and low-latency at the same time. Such edge platforms with multiple DNN models pose new challenges for scheduler designs. First, each request may have different service level objectives (SLOs) to improve quality of service (QoS). Second, the edge platforms should be able to efficiently schedule multiple heterogeneous DNN models so that system utilization can be improved. To meet these two goals, this paper proposes BCEdge, a novel learning-based scheduling framework that takes adaptive batching and concurrent execution of DNN inference services on edge platforms. We define a utility function to evaluate the trade-off between throughput and latency. The scheduler in BCEdge leverages maximum entropy-based deep reinforcement learning (DRL) to maximize utility by 1) co-optimizing batch size and 2) the number of concurrent models automatically. Our prototype implemented on different edge platforms shows that the proposed BCEdge enhances utility by up to 37.6% on average, compared to state-of-the-art solutions, while satisfying SLOs.</p>

show abstract

Section: A Model-level Dnn Inference Servicementioning

confidence: 99%

BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms

Zhang¹,

Li²,

Zhao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…A third approach reported in the literature proposes to delegate the computation of critical neurons to the TEE. Hou et al [17] propose a framework to design a secure version of the ML model with crafted random weights. More specifically, the framework adds crafted values to obfuscate strategical weights within the ANN.…”

Section: Selected Neuronsmentioning

confidence: 99%

“…Works that delegate the computation of selected neurons to the TEE are the most efficient in terms of TEE memory footprint. The work [17] only requires space to save the selected neurons and the output of the layer under process. Despite performing most of the computation outside the TEE, this work still requires the outputs of each layer to be denoised within the TEE, which significantly increases the number of context switches compared to our framework.…”

Section: Gap Analysismentioning

confidence: 99%

“…However, TEEs are designed to execute small critical operations and tend to pose severe memory constraints to resource-hungry ML computation [15]- [20]. Furthermore, they do not extend to accelerators such as GPUs or ASICs [15], [17]- [19], [21]. As a consequence, developing privacy-enhanced ML frameworks on top of TEEs remains an open challenge.…”

Section: Introductionmentioning

confidence: 99%

“…Despite the novelty, these approaches suffer from increased latency disrupted by the decryption of all ANN parameters and the frequent context switch from the secure-world to the normal-world. To counteract these issues, some works propose to (i) perform loading in parallel with prediction [20]; (ii) execute only sensitive layers within the enclave [16], [19], [21]; (iii) obfuscate only relevant ANN weights [17]; or (iv) refactor a model in a bident structure [24].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SecureQNN: Introducing a Privacy-Preserving Framework for QNNs at the Deep Edge

Costa,

Gomes,

Cabral

et al. 2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Recent concerns about real-time inference and data privacy are making Machine Learning (ML) shift to the edge. However, training efficient ML models require large-scale datasets not available for typical ML clients. Consequently, the training is usually delegated to specific Service Providers (SP), which are now worried to deploy proprietary ML models on untrusted edge devices. A natural solution to increase the privacy and integrity of ML models comes from Trusted Execution Environments (TEEs), which provide hardware-based security. However, their integration with heavy ML computation remains a challenge. This perspective paper explores the feasibility of leveraging a state-of-the-art TEE technology widely available in modern MCUs (TrustZone-M) to protect the privacy of Quantized Neural Networks (QNNs). We propose a novel framework that traverses the model layer-by-layer and evaluates the number of epochs an attacker requires to build a model with the same accuracy as the target with the information disclosed. The set of layers whose information makes the attacker spend less training effort than the owner training from scratch is protected in an isolated environment, i.e., the secure-world. Our framework will be evaluated in terms of latency and memory footprint for two ANNs built for the CIFAR-10 and Visual Wake Words (VWW) datasets. In this perspective paper, we establish a baseline reference for the results.

show abstract

Privacy-Preserving Data Analytics in Usage-Based Insurance

Huang,

Shen

2024

Wireless Networks

View full text Add to dashboard Cite

Model Protection: Real-Time Privacy-Preserving Inference Service for Model Privacy at the Edge

Cited by 21 publications

References 29 publications

BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms

BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms

SecureQNN: Introducing a Privacy-Preserving Framework for QNNs at the Deep Edge

Privacy-Preserving Data Analytics in Usage-Based Insurance

Contact Info

Product

Resources

About