Differentially Private Learning with Per-Sample Adaptive Clipping

Xia, Tianyu; Shen, Shuheng; Yao, Su; Fu, Xiaoyi; Xu, Ke; Xu, Xiaolong; Xing, Feifei; Wang, Weiqiang

doi:10.48550/arxiv.2212.00328

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The gradients are divided into different groups and each group is clipped separately. Xia et al [13] propose Differentially Private Per-Sample Adaptive Clipping (DP-PSAC) algorithm. It coupled the clipping threshold with the learning rate to avoid tuning the clipping threshold.…”

Section: Related Workmentioning

confidence: 99%

“…To verify the effectiveness of the AFRRS mechanism, the experiments were designed to compare TSA [10], BC [11], DPL-GGC [12], DP-PSAC [13], AUTO clipping [14], ADPSGD [15], DPNASNet [16], and the deep learning model without differential privacy protection (No DP). The results are shown in Table 3 and Table 4.…”

Section: Effectiveness Evaluation Of Different Algorithmsmentioning

confidence: 99%

“…Algorithm Accuracy (%) PB ( ) No DP 99.00 -TSA [10] 98.10 2.93 DPL-GGC [12] 98.23 2.85 DP-PSAC [13] 98.24 3.00 AUTO clipping [14] 98.15 3.00 DPNASNet [16] 98.57 3.00 AFRRS 98.80 0.60…”

Section: Tablementioning

confidence: 99%

See 2 more Smart Citations

Preserving Differential Privacy in Deep Learning Based on Feature Relevance Region Segmentation

Wang

Xie

Tan

et al. 2024

IEEE Trans. Emerg. Topics Comput.

View full text Add to dashboard Cite

In the era of big data, deep learning techniques provide intelligent solutions for various problems in real-life scenarios. However, deep neural networks depend on large-scale datasets including sensitive data, which causes the potential risk of privacy leakage. In addition, various constantly evolving attack methods are also threatening the data security in deep learning models. Protecting data privacy effectively at a lower cost has become an urgent challenge. This paper proposes an Adaptive Feature Relevance Region Segmentation (AFRRS) mechanism to provide differential privacy preservation. The core idea is to divide the input features into different regions with different relevance according to the relevance between input features and the model output. Less noise is intentionally injected into the region with stronger relevance, and more noise is injected into the regions with weaker relevance. Furthermore, we perturb loss functions by injecting noise into the polynomial coefficients of the expansion of the objective function to protect the privacy of data labels. Theoretical analysis and experiments have shown that the proposed AFRRS mechanism can not only provide strong privacy preservation for the deep learning model, but also maintain the good utility of the model under a given moderate privacy budget compared with existing methods.

show abstract

Section: Related Workmentioning

confidence: 99%