InvBERT: Text Reconstruction from Contextualized Embeddings used for Derived Text Formats of Literary Works

Kugler, Kai; Münker, Simon; Hohmann, J.; Rettinger, Achim

doi:10.48550/arxiv.2109.10104

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…KNN (Qu et al, 2021) selects the closest word in the embedding space as the real word. InvBert (Höhmann et al, 2021) trains an embedding inversion model that takes word representations as input and outputs each representation's original word one-to-one. MLC (Song and Raghunathan, 2020) is similar to InvBert, the difference is that MLC performs multi-label classification tasks, predicting all the words that have appeared without emphasizing the order.…”

Section: Baselinesmentioning

confidence: 99%

“…For example, they can use these data to train a better model or extract users' private information, such as personal details and confidential business information, even if it is prohibited by law. Recent literature (Song and Shmatikov, 2019;Pan et al, 2020) shows that even uploading word representations can still leak privacy, as the embedding inversion attack (Höhmann et al, 2021) can restore word representations to their original words.…”

Section: Plaintext Privacy During Inferencementioning

confidence: 99%

“…Many sought to make users query PLM services with word representations rather than plaintext. But digital word representations still need additional privacy protection because recent studies (Pan et al, 2020;Song and Raghunathan, 2020) have revealed that standard word representation can be easily restored to its original word under embedding inversion attacks (Höhmann et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference

Zhou,

Lu,

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Pre-trained language models (PLMs) are often deployed as cloud services, enabling users to upload textual data and perform inference remotely. However, users' personal text often contains sensitive information, and sharing such data directly with the service providers can lead to serious privacy leakage. To address this problem, we introduce a novel privacy-preserving inference framework called TextMixer, which prevents plaintext leakage during the inference phase. Inspired by k-anonymity, TextMixer aims to obfuscate a user's private input by mixing it with multiple other inputs, thereby confounding potential privacy attackers. To achieve this, our approach involves: (1) proposing a novel encryption module, Privacy Mixer, which encrypts input from three distinct dimensions: mixing, representation, and position. ( 2) adopting a pre-trained Multi-input Multi-output network to handle mixed representations and obtain multiple predictions. (3) employing a Privacy Demixer to ensure only the user can decrypt the real output among the multiple predictions. Furthermore, we explore different ways to automatically generate synthetic inputs required for mixing. Experimental results on token and sentence classification tasks demonstrate that TextMixer greatly surpasses existing privacy-preserving methods in both performance and privacy.

show abstract

Section: Baselinesmentioning

confidence: 99%