A theory of independent mechanisms for extrapolation in generative models

Besserve, Michel; Sun, Rémy; Janzing, Dominik; Schölkopf, Bernhard

doi:10.48550/arxiv.2004.00184

Cited by 3 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extensive results in ZSL and OSR show that our method indeed improves the balance and hence achieves the state-ofthe-art performance. As future direction, we will seek new definitions on disentanglement [58] and devise practical implementations to achieve improved disentanglement [9]. A.1.…”

Section: Discussionmentioning

confidence: 99%

“…For example, recent GAN models [74,26] can generate a large variety of photorealistic images by only using Z. In fact, as shown in recent literature [9], it is possible for an over-parameterized model P θ (X|Z, Y ) to ignore Y and use purely Z to generate X, leading to non-faithful generations. This is because the information in Y might be fully contained in Z.…”

Section: Counterfactual-faithful Trainingmentioning

confidence: 99%

“…To the best of our knowledge, the proposed counterfactual framework is the first to provide a theoretical ground for balancing and improving the seen/unseen classification. In particular, we show that the quality of disentangling Z and Y is the key bottleneck, so it is a potential future direction for ZSL/OSR [58,19,46,9]. Our method can serve as an unseen/seen binary classifier, which can plug-and-play and boost existing ZSL/OSR methods to achieve new stateof-the-arts (Section 4).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Counterfactual Zero-Shot and Open-Set Visual Recognition

Yue¹,

Wang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a novel counterfactual framework for both Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR), whose common challenge is generalizing to the unseen-classes by only training on the seen-classes. Our idea stems from the observation that the generated samples for unseen-classes are often out of the true distribution, which causes severe recognition rate imbalance between the seen-class (high) and unseen-class (low). We show that the key reason is that the generation is not Counterfactual Faithful, and thus we propose a faithful one, whose generation is from the sample-specific counterfactual question: What would the sample look like, if we set its class attribute to a certain class, while keeping its sample attribute unchanged? Thanks to the faithfulness, we can apply the Consistency Rule to perform unseen/seen binary classification, by asking: Would its counterfactual still look like itself? If "yes", the sample is from a certain class, and "no" otherwise. Through extensive experiments on ZSL and OSR, we demonstrate that our framework effectively mitigates the seen/unseen imbalance and hence significantly improves the overall performance. Note that this framework is orthogonal to existing methods, thus, it can serve as a new baseline to evaluate how ZSL/OSR models generalize.Codes are available at https://github.com/yuezhongqi/gcm-cf.* Equal contribution 1 Conventional ZSL [31] only evaluates the recognition on the unseenclasses. We refer to ZSL as a more challenging setting: Generalized ZSL [12], which is evaluated on the both seen-and unseen-classes.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Counterfactual-faithful Trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Counterfactual Zero-Shot and Open-Set Visual Recognition

Yue¹,

Wang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Based on group theory and symmetry transformations, Higgins et al (2018) provides the "first principled definition of a disentangled representation". Closely related to this concept is also the field of causality in machine learning (Schölkopf, 2019;Suter et al, 2019), more specifically the search for causal generative models Besserve et al (2018Besserve et al ( , 2020. In terms of implementable metrics, a variety of quantities have been introduced, such as the β-VAE score Higgins et al (2017), SAP score Kumar et al (2017), DCI scores Eastwood and Williams (2018) and the Mutual Information Gap (MIG, Chen et al (2018)).…”

Section: Related Workmentioning

confidence: 99%

Demystifying Inductive Biases for $β$-VAE Based Architectures

Zietlow,

Rolinek,

Martius

2021

Preprint

View full text Add to dashboard Cite

The performance of β-Variational-Autoencoders (β-VAEs) and their variants on learning semantically meaningful, disentangled representations is unparalleled. On the other hand, there are theoretical arguments suggesting the impossibility of unsupervised disentanglement. In this work, we shed light on the inductive bias responsible for the success of VAE-based architectures. We show that in classical datasets the structure of variance, induced by the generating factors, is conveniently aligned with the latent directions fostered by the VAE objective. This builds the pivotal bias on which the disentangling abilities of VAEs rely. By small, elaborate perturbations of existing datasets, we hide the convenient correlation structure that is easily exploited by a variety of architectures. To demonstrate this, we construct modified versions of standard datasets in which (i) the generative factors are perfectly preserved; (ii) each image undergoes a mild transformation causing a small change of variance; (iii) the leading VAE-based disentanglement architectures fail to produce disentangled representations whilst the performance of a nonvariational method remains unchanged. The construction of our modifications is nontrivial and relies on recent progress on mechanistic understanding of β-VAEs and their connection to PCA. We strengthen that connection by providing additional insights that are of stand-alone interest.

show abstract

“…Note that the given "fact" is Z = z(x) and the To the best of our knowledge, the proposed counterfactual framework is the first to provide a theoretical ground for balancing and improving the seen/unseen classification. In particular, we show that the quality of disentangling Z and Y is the key bottleneck, so it is a potential future direction for ZSL/OSR [38,167,231,311]. Our method can serve as an unseen/seen binary classifier, which can plug-and-play and boost existing ZSL/OSR methods to achieve new state-of-the-arts (Section 5.2.6).…”

Section: Introductionmentioning

confidence: 96%

Causal view of generalization

Yue

View full text Add to dashboard Cite

First of all, I would like to thank my supervisor Prof. Hanwang Zhang for his hands-on guidance throughout my Ph.D. study. When I first started my Ph.D., I was a complete novice in research. He patiently taught me everything-from the grand scheme of things (e.g., idea formulation) to the nitty-gritty details (e.g., best practice to format a LaTeX project). When things got tough and I thought about quitting, he was right by my side and motivated me to keep going. Even when I encounter personal difficulties, he was compassionate and offered tremendous support that helped me cope. Thanks to him, I gained so much in this journey and I can't wait to climb the next mountain.I would like to thank my collaborators Ph.

show abstract

A theory of independent mechanisms for extrapolation in generative models

Cited by 3 publications

References 13 publications

Counterfactual Zero-Shot and Open-Set Visual Recognition

Counterfactual Zero-Shot and Open-Set Visual Recognition

Demystifying Inductive Biases for $β$-VAE Based Architectures

Causal view of generalization

Contact Info

Product

Resources

About