Knowledge Unlearning for Mitigating Privacy Risks in Language Models

Abstract

Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities. Previous work addressing privacy issues for language models has mostly focused on data preprocessing and differential privacy methods, both requiring re-training the underlying LM. We propose knowledge unlearning as an alternative method to reduce privacy risks for LMs post hoc. We show that simply applying the unlikelihood training objective to target token sequences is effective at forgetting them with little to no degradation of general language modeling performances; it sometimes even substantially improves the underlying LM with just a few iterations. We also find that sequential unlearning is better than trying to unlearn all the data at once and that unlearning is highly dependent on which kind of data (domain) is forgotten. By showing comparisons with a previous data preprocessing method known to mitigate privacy risks for LMs, we show that unlearning can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being orders of magnitude more computationally efficient. We release the code and dataset needed to replicate our results at https://github.com/joeljang/knowledge-unlearning.

Anatomy of Learned Database Tuning with Bayesian Optimization

“…To mitigate the costs associated with retraining LLMs for each forget request, approximate unlearning methods have been proposed (Pawelczyk et al, 2023;Chen & Yang, 2023;Meng et al, 2022;Jang et al, 2022;Kassem et al, 2023) which remove the contribution of the target data points from the pretrained model.…”

“…The current SOTA approach for our task at hand (also regarded as SOTA in (Liu & Kalinli, 2023;Kassem et al, 2023;Pawelczyk et al, 2023) is the work of (Jang et al, 2022), who uses gradient ascent (GA) oblivious to the pretraining dataset to minimize exact memorization over a set of textual sequences. It shows that simply maximizing the loss with respect to the textual sequences to be forgotten produces a competitive trade-off between privacy and model utility.…”

“…To quantify the dynamics of memorization during unlearning we adopt a variation of label memorization (Zhang et al, 2021a;Pondenkandath et al, 2018) produced by (Tirumala et al, 2022) to study the properties of neural LMs during training, referred to as exact memorization. The motivation for using exact memorization to characterise LLMs' knowledge over a textual sequence is twofold: 1) it is prominent in previous work that explores copyright issues in LLMs (Jang et al, 2022;Tirumala et al, 2022), and 2) it works well in practical settings as it is inexpensive to compute.…”

“…data points, text sequences) to be forgotten, and, on the other hand, on the capacity (number of parameters) of the model. Existing state-of-the-art solutions include self-supervised learning techniques (Jang et al, 2022;Liu & Kalinli, 2023;Chen & Yang, 2023) and reinforcement learning methods (Kassem et al, 2023;Lu et al, 2022). To quantify the success of unlearning, previous methods holistically characterize the memorization of the whole forget set, that is, the set of textual sequences that are to be 'unlearned'.…”

“…To quantify the success of unlearning, previous methods holistically characterize the memorization of the whole forget set, that is, the set of textual sequences that are to be 'unlearned'. Hence, the forget set is given a "memorization score" which is the average of the memorization scores of each textual sequence (Jang et al, 2022). If the average score is within an expected threshold, then the unlearning algorithm is deemed successful.…”