Locally Differentially Private Sparse Vector Aggregation

Zhou, Mingxun; Wang, Tianhao; Chan, T.-H. Hubert; Fanti, Giulia; Shi, Elaine

doi:10.1109/sp46214.2022.9833635

Cited by 13 publications

(1 citation statement)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5 In contrast to the general normalization, it aims for weaker SeqLDP at the token level (cf. event-level vs. userlevel LDP [81]), a finer granularity in the "protection hierarchy," protecting any neighboring sequences (vs. datasets) differing in any single token (vs. sequence). Details are deferred to Appendix A.…”

Section: Sequence Local Dp (Seqldp)mentioning

confidence: 99%

DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass

Du,

Yue,

Chow

et al. 2023

Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

Self Cite

View full text Add to dashboard Cite

Differentially private stochastic gradient descent (DP-SGD) adds noise to gradients in back-propagation, safeguarding training data from privacy leakage, particularly membership inference. It fails to cover (inference-time) threats like embedding inversion and sensitive attribute inference. It is also costly in storage and computation when used to fine-tune large pre-trained language models (LMs).We propose DP-Forward, which directly perturbs embedding matrices in the forward pass of LMs. It satisfies stringent local DP requirements for training and inference data. To instantiate it using the smallest matrix-valued noise, we devise an analytic matrix Gaussian mechanism (aMGM) by drawing possibly non-i.i.d. noise from a matrix Gaussian distribution. We then investigate perturbing outputs from different hidden (sub-)layers of LMs with aMGM noises. Its utility on three typical tasks almost hits the non-private baseline and outperforms DP-SGD by up to 7.7pp at a moderate privacy level. It saves 3× time and memory costs compared to DP-SGD with the latest high-speed library. It also reduces the average success rates of embedding inversion and sensitive attribute inference by up to 88pp and 41pp, respectively, whereas DP-SGD fails. CCS CONCEPTS• Security and privacy → Data anonymization and sanitization; Privacy-preserving protocols; Privacy protections.

show abstract

Section: Sequence Local Dp (Seqldp)mentioning

confidence: 99%