Rényi Fair Information Bottleneck for Image Classification

Gronowski, Adam; Paul, William; Alajaji, Fady; Gharesifard, Bahman; Burlina, Philippe

doi:10.1109/cwit55308.2022.9817669

Cited by 4 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…for α > 0, α = 1 and distributions P and Q with common support X . 1 Using Rényi divergence gives an extra degree of freedom and allows more control over the compression term I(X; Z). As the Rényi divergence is non-decreasing with α, a higher α will more strongly force the distribution P Z|X closer to Q Z , resulting in more compression.…”

Section: Variational Boundsmentioning

confidence: 99%

See 1 more Smart Citation

Achieving Utility, Fairness, and Compactness via Tunable Information Bottleneck Measures

Gronowski¹,

Paul²,

Alajaji³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Designing machine learning algorithms that are accurate yet fair, not discriminating based on any sensitive attribute, is of paramount importance for society to accept AI for critical applications. In this article, we propose a novel fair representation learning method termed the Rényi Fair Information Bottleneck Method (RFIB) which incorporates constraints for utility, fairness, and compactness of representation, and apply it to image classification. A key attribute of our approach is that we consider -in contrast to most prior work -both demographic parity and equalized odds as fairness constraints, allowing for a more nuanced satisfaction of both criteria. Leveraging a variational approach, we show that our objectives yield a loss function involving classical Information Bottleneck (IB) measures and establish an upper bound in terms of the Rényi divergence of order α on the mutual information IB term measuring compactness between the input and its encoded embedding. Experimenting on three different image datasets (EyePACS, CelebA, and FairFace), we study the influence of the α parameter as well as two other tunable IB parameters on achieving utility/fairness trade-off goals, and show that the α parameter gives an additional degree of freedom that can be used to control the compactness of the representation. We evaluate the performance of our method using various utility, fairness, and compound utility/fairness metrics, showing that RFIB outperforms current state-of-the-art approaches.

show abstract

Section: Variational Boundsmentioning

confidence: 99%

“…To compute the bounds in practice we use the reparameterization trick [56]. Modeling P Z|X as a density, we let P Z|X dZ = P E dE, where E is a random variable and Z = f (X, E) is a deterministic function, allowing us to backpropagate gradients 1 If P and Q are probability density functions, then Dα(P ||Q)…”

Section: Computing the Boundsmentioning

confidence: 99%

Achieving Utility, Fairness, and Compactness via Tunable Information Bottleneck Measures

Gronowski¹,

Paul²,

Alajaji³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In supervised learning, the information bottleneck maximizes the compression of redundant information between the input and the output while preserving the mutual information about the desired output. Based on this concept, researchers have proposed a range of machine learning algorithms that have been extensively used in various fields including image, speech, and natural language processing [2][3][4][5] .…”

Section: Introductionmentioning

confidence: 99%

Deep Gaussian Mixture Variational Information Bottleneck

Zuo¹

2023

AETR

View full text Add to dashboard Cite

On supervised learning tasks, introducing an information bottleneck can guide the model to focus on the more discriminative features in the input, which can effectively prevent overfitting. The deep variational information bottleneck aims to learn a global Gaussian latent variable using the neural network, which compresses mutual information with the input features and retains the most relevant information with the output features as much as possible. However, for the input containing complex semantic information, such as multi-label classification datasets, the latent variable obeying a simple Gaussian distribution may not necessarily capture rich representations. To alleviate this drawback, in this paper, we propose a new approach called Gaussian Mixture Variational Information Bottleneck (GMVIB) where the latent variable follows Gaussian mixture distribution. Then we generate Multi-MNIST and Multi-FashionMNIST, the multi-label classification datasets based on MNIST and FashionMNIST. Our experiments on these datasets show that the proposed approach learns a more efficient embedding representation and achieves competitive performance.

show abstract

Addressing fairness issues in deep learning-based medical image analysis: a systematic review

Xu,

Li,

Yao

et al. 2024

npj Digit. Med.

View full text Add to dashboard Cite

Deep learning algorithms have demonstrated remarkable efficacy in various medical image analysis (MedIA) applications. However, recent research highlights a performance disparity in these algorithms when applied to specific subgroups, such as exhibiting poorer predictive performance in elderly females. Addressing this fairness issue has become a collaborative effort involving AI scientists and clinicians seeking to understand its origins and develop solutions for mitigation within MedIA. In this survey, we thoroughly examine the current advancements in addressing fairness issues in MedIA, focusing on methodological approaches. We introduce the basics of group fairness and subsequently categorize studies on fair MedIA into fairness evaluation and unfairness mitigation. Detailed methods employed in these studies are presented too. Our survey concludes with a discussion of existing challenges and opportunities in establishing a fair MedIA and healthcare system. By offering this comprehensive review, we aim to foster a shared understanding of fairness among AI researchers and clinicians, enhance the development of unfairness mitigation methods, and contribute to the creation of an equitable MedIA society.

show abstract

Rényi Fair Information Bottleneck for Image Classification

Cited by 4 publications

References 16 publications

Achieving Utility, Fairness, and Compactness via Tunable Information Bottleneck Measures

Achieving Utility, Fairness, and Compactness via Tunable Information Bottleneck Measures

Deep Gaussian Mixture Variational Information Bottleneck

Addressing fairness issues in deep learning-based medical image analysis: a systematic review

Contact Info

Product

Resources

About