On the Privacy–Utility Trade-Off in Differentially Private Hierarchical Text Classification

Abstract

Hierarchical text classification consists of classifying text documents into a hierarchy of classes and sub-classes. Although Artificial Neural Networks have proved useful to perform this task, unfortunately, they can leak training data information to adversaries due to training data memorization. Using differential privacy during model training can mitigate leakage attacks against trained models, enabling the models to be shared safely at the cost of reduced model accuracy. This work investigates the privacy–utility trade-off in hierarchical text classification with differential privacy guarantees, and it identifies neural network architectures that offer superior trade-offs. To this end, we use a white-box membership inference attack to empirically assess the information leakage of three widely used neural network architectures. We show that large differential privacy parameters already suffice to completely mitigate membership inference attacks, thus resulting only in a moderate decrease in model utility. More specifically, for large datasets with long texts, we observed Transformer-based models to achieve an overall favorable privacy–utility trade-off, while for smaller datasets with shorter texts, convolutional neural networks are preferable.

Towards Privacy Preserved Document Image Classification - A Comprehensive Benchmark

“…In addition, the unintentional memorization [17] of training samples in these models could directly expose information about the training dataset. Surprisingly, while a plethora of research has been conducted on both document classification [5,6,35] and privacy in textual documents [13,14,18,30], we found no existing literature in the field that addresses the issue of data privacy and potential information leakage from AI-powered document image classification systems. In this work, therefore, we investigate the potential of latest privacy preservation techniques [22,23,26,28] in combination with state-of-the-art DL-based document image classification models to assess whether they can achieve sufficient utility under strong privacy constraints.…”

“…They further demonstrated that MIAs can be successfully applied to DL models, even with only a black-box access to the target model, achieved by training multiple shadow models that mimic the target model. A number of derivative works have further explored membership inference attacks in other tasks [10,13,14].…”

“…The rapid evolution of artificial intelligence (AI), notably in computer vision [1,2] and natural language processing [3,4], has revolutionized the field of document analysis, with modern Deep Learning (DL)-based systems delivering superhuman performances across a row of document understanding tasks [5][6][7][8][9]. Simultaneously, however, concerns have been raised about potential privacy breaches and the inadvertent leakage of sensitive user data through the widespread use of such data-driven AI systems [10][11][12][13][14][15].…”

“…Numerous recent studies [12,[15][16][17] have shown that, when not trained with rigorous privacy constraints, deep learning (DL) models can readily become sources of information leakage. Reconstruction of the training data statistics [15,18], inferring whether a sample comes from the training data distribution [12][13][14], or model stealing [19,20] are just a few examples of potential privacy violations. If deep models with such vulnerabilities were to be directly trained on private document data, which often contains sensitive information-such as names, addresses, contact details, social security numbers, financial particulars, and, most critically, an organization's intellectual property-they could potentially be exploited, leading to significant harm to individuals or organizations.…”

“…To address the aforementioned privacy challenges in AI-powered systems, a number of privacy-preserving approaches have been recently developed [22][23][24][25][26][27][28]. Most notably, Differential Privacy (DP) [22], Federated Learning (FL) [23,24], have demonstrated promising results across various application domains such as medical imaging [29], time series analysis [26], and natural language processing (NLP) [13,14,18,30]. In the context of document AI, such privacy techniques may be applied under different settings.…”