Variational approach for privacy funnel optimization on continuous data

Nan, Lihao; Tao, Dacheng

doi:10.1016/j.jpdc.2019.09.010

Cited by 5 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the state-of-the-art techniques for fairness penalty computations depend on mutual information-related measures [31,28,26]. These information-theoretic methods achieve fairness at the expense of data quality and utility.…”

Section: Fairness Objectivementioning

confidence: 99%

Fair Latent Deep Generative Models (FLDGMs) for Syntax-Agnostic and Fair Synthetic Data Generation

Ramachandranpillai,

Sikder,

Heintz

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Deep Generative Models (DGMs) for generating synthetic data with properties such as quality, diversity, fidelity, and privacy is an important research topic. Fairness is one particular aspect that has not received the attention it deserves. One difficulty is training DGMs with an in-process fairness objective, which can disturb the global convergence characteristics. To address this, we propose Fair Latent Deep Generative Models (FLDGMs) as enablers for more flexible and stable training of fair DGMs, by first learning a syntax-agnostic, model-agnostic fair latent representation (low dimensional) of the data. This separates the fairness optimization and data generation processes thereby boosting stability and optimization performance. Moreover, data generation in the low dimensional space enhances the accessibility of models by reducing computational demands. We conduct extensive experiments on image and tabular domains using Generative Adversarial Networks (GANs) and Diffusion Models (DMs) and compare them to the state-of-the-art in terms of fairness and utility. Our proposed FLDGMs achieve superior performance in generating high-quality, high-fidelity, and high-diversity fair synthetic data compared to the state-of-the-art fair generative models.

show abstract

Section: Fairness Objectivementioning

confidence: 99%

Fair Latent Deep Generative Models (FLDGMs) for Syntax-Agnostic and Fair Synthetic Data Generation

Ramachandranpillai,

Sikder,

Heintz

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…Privacy-preserving deep learning [21,4,8] involves learning representations that incorporate features from the data relevant to the given task and ignore sensitive information (such as the identity of a person). The authors in [23] propose a simple variational approach for privacy-preserving representation learning. In contrast to existing privacy preservation works, the objective of the RCRMR-LD problem setting is to achieve class-level forgetting, i.e., if a class is declared as private/restricted, then all information about this class should be removed from the model trained on it, without affecting its ability to identify the remaining classes.…”

Section: Related Workmentioning

confidence: 99%

Attaining Class-Level Forgetting in Pretrained Model Using Few Samples

Singh

Mazumder

Asad

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In order to address real-world problems, deep learning models are jointly trained on many classes. However, in the future, some classes may become restricted due to privacy/ethical concerns, and the restricted class knowledge has to be removed from the models that have been trained on them. The available data may also be limited due to privacy/ethical concerns, and re-training the model will not be possible. We propose a novel approach to address this problem without affecting the model's prediction power for the remaining classes. Our approach identifies the model parameters that are highly relevant to the restricted classes and removes the knowledge regarding the restricted classes from them using the limited available training data. Our approach is significantly faster and performs similar to the model re-trained on the complete data of the remaining classes.

show abstract

“…For this reason, approaches that take advantage of the scalability of deep learning have emerged. For instance, in [9] they learn the representations through adversarial learning, while in the the privacy preserving variational autoencoder (PPVAE) [17] and the unsupervised version of the variational fair autoencoder (VFAE) [27] they learn such representations with variational inference.…”

Section: Privacymentioning

confidence: 99%

“…Finally, the resulting approaches for privacy and fairness can be implemented with little modification to common algorithms for representation learning like the variational autoencoder (VAE) [15], the β-VAE [16], the variational information bottleneck (VIB) [17], or the nonlinear information bottleneck [18]. Therefore, it facilitates the incorporation of private and fair representations in current applications (see the supplementary material B for a guide on how to modify these algorithms).…”

Section: Introductionmentioning

confidence: 99%

A Variational Approach to Privacy and Fairness

Rodríguez-Gálvez¹,

Thobaben²,

Skoglund³

2020

Preprint

View full text Add to dashboard Cite

In this article, we propose a new variational approach to learn private and/or fair representations. This approach is based on the Lagrangians of a new formulation of the privacy and fairness optimization problems that we propose. In this formulation, we aim at generating representations of the data that keep a prescribed level of the relevant information that is not shared by the private or sensitive data, while minimizing the remaining information they keep. The proposed approach (i) exhibits the similarities of the privacy and fairness problems, (ii) allows us to control the trade-off between utility and privacy or fairness through the Lagrange multiplier parameter, and (iii) can be comfortably incorporated to common representation learning algorithms such as the VAE, the β-VAE, the VIB, or the nonlinear IB.Preprint. Under review.

show abstract

Variational approach for privacy funnel optimization on continuous data

Cited by 5 publications

References 7 publications

Fair Latent Deep Generative Models (FLDGMs) for Syntax-Agnostic and Fair Synthetic Data Generation

Fair Latent Deep Generative Models (FLDGMs) for Syntax-Agnostic and Fair Synthetic Data Generation

Attaining Class-Level Forgetting in Pretrained Model Using Few Samples

A Variational Approach to Privacy and Fairness

Contact Info

Product

Resources

About