Generative Interventions for Causal Learning

Mao, Chengzhi; Cha, Augustine; Gupta, Amogh; Yang, Junfeng; Vondrick, Carl

doi:10.1109/cvpr46437.2021.00394

Cited by 40 publications

(28 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most cases, we find that GAN transformations that modify style tend to be more beneficial than those that modify poses. This is in line with previous works that note the benefits of style-based training augmentations for image classification [41,14] and related positional sensitivities of classifiers [71,3,16]. In the more difficult Imagenet classification problem, we found performance degrades substantially during image projection, and therefore GAN perturbations offer limited benefits.…”

Section: Discussionsupporting

confidence: 91%

“…Here, we choose the StyleGAN2 generator [31], which focus on modeling a single object category (such as faces, cars, and cats), rather than class-conditional models like BigGAN [7] on 1000 ImageNet categories [53]. Due to the higher variation in multi-class data, image projection (a critical part of generating deep augmentations) is still not sufficiently fast and reliable across the full dataset [26,43,41]. Even within a single-class StyleGAN2 generator, some aspects of the original image cannot be accurately recovered, such as fine textures, ornate backgrounds, or non-canonical poses (shown in supplementary material), impacting downstream classification.…”

Section: Discussionmentioning

confidence: 99%

“…Outputs from a generative model can also be used in adversarial settings: Samangouei et al [55] demonstrate projecting adversarial images through a generator on MNIST and Fashion-MNIST, while other works [57,28,20] train models using the generator to improve robustness. Augmentation using GANs can be applied at training time for robustness or test-time for ensembling; while we primarily focus on the latter, concurrent works [41,56] investigate the benefits of the former approach while [62] investigates intermediate GAN representations for few-shot segmentation.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Ensembling with Deep Generative Views

Chai¹,

Zhu²,

Shechtman³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose, simply by learning from unlabeled image collections. Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification. Using a pretrained generator, we first find the latent code corresponding to a given real input image. Applying perturbations to the code creates natural variations of the image, which can then be ensembled together at test-time. We use StyleGAN2 as the source of generative augmentations and investigate this setup on classification tasks involving facial attributes, cat faces, and cars. Critically, we find that several design decisions are required towards making this process work; the perturbation procedure, weighting between the augmentations and original image, and training the classifier on synthesized images can all impact the result. Currently, we find that while test-time ensembling with GAN-based augmentations can offer some small improvements, the remaining bottlenecks are the efficiency and accuracy of the GAN reconstructions, coupled with classifier sensitivities to artifacts in GAN-generated images.

show abstract

Section: Discussionsupporting

confidence: 91%

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Ensembling with Deep Generative Views

Chai¹,

Zhu²,

Shechtman³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Pre-Trained GANs for Vision. Prior work has explored the use of GANs [27,65] in vision tasks such as classification [10,12,54,71,81], segmentation [55,76,79,87] and representation learning [7,20,21,23,35]. Likewise, we share the goal of leveraging the power of pre-trained deep generative models for vision tasks.…”

Section: Related Workmentioning

confidence: 99%

GAN-Supervised Dense Visual Alignment

Peebles¹,

Zhu²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Figure 1. Given an input dataset of unaligned images, our GANgealing algorithm discovers dense correspondences between all images. Top row: Images from LSUN Cats and the dataset's average image. Second row: Our learned transformations of the input images. Third row: Dense correspondences learned by GANgealing. Bottom row: By annotating the average transformed image, we can propagate user edits to images and videos. Please see our project page for detailed video results: www.wpeebles.com/gangealing.

show abstract

“…causes). When combined with statistical modeling (i.e., regression), one can estimate the functional relationship between interventions on causes and effects [Pearl, 2009a, Shalizi, 2013, Shi et al, 2019, Mao et al, 2020.…”

Section: Introductionmentioning

confidence: 99%

Causal inference for process understanding in Earth sciences

Massmann,

Gentine,

Runge

2021

Preprint

View full text Add to dashboard Cite

There is growing interest in the study of causal methods in the Earth sciences. However, most applications have focused on causal discovery, i.e. inferring the causal relationships and causal structure from data. This paper instead examines causality through the lens of causal inference and how expert-defined causal graphs, a fundamental from causal theory, can be used to clarify assumptions, identify tractable problems, and aid interpretation of results and their causality in Earth science research. We apply causal theory to generic graphs of the Earth system to identify where causal inference may be most tractable and useful to address problems in Earth science, and avoid potentially incorrect conclusions. Specifically, causal inference may be useful when: (1) the effect of interest is only causally affected by the observed portion of the state space; or: (2) the cause of interest can be assumed to be independent of the evolution of the system's state; or: (3) the state space of the system is reconstructable from lagged observations of the system. However, we also highlight through examples that causal graphs can be used to explicitly define and communicate assumptions and hypotheses, and help to structure analyses, even if causal inference is ultimately challenging given the data availability, limitations and uncertainties.

show abstract

Generative Interventions for Causal Learning

Cited by 40 publications

References 14 publications

Ensembling with Deep Generative Views

Ensembling with Deep Generative Views

GAN-Supervised Dense Visual Alignment

Causal inference for process understanding in Earth sciences

Contact Info

Product

Resources

About