Towards Privacy Preserved Document Image Classification - A Comprehensive Benchmark

Abstract

As data-driven AI systems become increasingly integrated into industry, concerns have recently arisen regarding potential privacy breaches and the inadvertent leakage of sensitive user data through the exploitation of these systems. In this paper, we explore the intersection of data privacy and AI-powered document analysis systems, presenting a comprehensive benchmark of well-known privacy-preserving methods for the task of document image classification. In particular, we investigate four different privacy methods---Differential Privacy (DP), Federated Learning (FL), Differentially Private Federated Learning (DP-FL), and Secure Multi-Party Computation (SMPC)---on two well-known document benchmark datasets, namely RVL-CDIP and Tobacco3482. Furthermore, we investigate the performance of each method under a variety of configurations for thorough benchmarking. Finally, the privacy strength of each approach is assessed by subjecting the private models to well-known membership inference attacks. Our results demonstrate that, with sufficient tuning of hyperparameters, Differential Privacy (DP) can achieve reasonable performance on the task of document image classification while also ensuring rigorous privacy constraints, both in standalone and federated learning setups. On the other hand, while FL-based approaches present less implementation complexity and incur little to no loss in performance on the task, they do not offer sufficient protection against privacy attacks. By rigorously benchmarking various privacy approaches, our study paves the way for integrating deep document classification models into industrial pipelines while meeting regulatory and ethical standards, including GDPR and the AI Act 2022.

Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI

“…However, recent work suggests that methods for privacy-preserving machine learning also have an influence on the quality of explanations. While Franco et al (2021) were among the first to combine privacy and explanation methods, Saifullah et al (2022) and Bozorgpanah et al (2022) were the first to provide extensive analyses on the impact of privacy-preserving methods on XAI methods. Saifullah et al (2022) investigated the impact of different privacy-preserving methods on attribution-based explanations in different domains including time-series, document image, and medical image analysis.…”

“…While Franco et al (2021) were among the first to combine privacy and explanation methods, Saifullah et al (2022) and Bozorgpanah et al (2022) were the first to provide extensive analyses on the impact of privacy-preserving methods on XAI methods. Saifullah et al (2022) investigated the impact of different privacy-preserving methods on attribution-based explanations in different domains including time-series, document image, and medical image analysis. Their study suggests that different privacy methods have different effects on the quality of attribution-based explanations, and that perturbation-based XAI methods are less affected by noise introduced through differential privacy.…”

“…Several recent studies revealed that both explainability and privacy affect each other significantly in practical deployment. While many works ( Milli et al, 2019 ; Aïvodji et al, 2020 ; Shokri et al, 2021 ; Duddu and Boutet, 2022 ) found that revealing explanations can pose a severe security risk, others ( Bozorgpanah et al, 2022 ; Saifullah et al, 2022 ) found that adding privacy can significantly diminish the quality of attribution maps.…”

“…However, their increasing relevance raises important questions about the impact they have on the privacy of an AI system. Most previous works investigating the interdependency of privacy and explainability focused on low-level XAI methods based on input feature attribution ( Milli et al, 2019 ; Shokri et al, 2021 ; Saifullah et al, 2022 ), while Montenegro et al (2022) investigated privacy-preserving case-based explanations. We argue that more complex XAI methods have received too little attention when it comes to an assessment of their implication on privacy.…”