Generative Bias for Visual Question Answering

Abstract

The task of Visual Question Answering (VQA) is known to be plagued by the issue of VQA models exploiting biases within the dataset to make its final prediction. Many previous ensemble based debiasing methods have been proposed where an additional model is purposefully trained to be biased in order to aid in training a robust target model. However, these methods compute the bias for a model from the label statistics of the training data or directly from single modal branches. In contrast, in this work, in order to better learn the bias a target VQA model suffers from, we propose a generative method to train the bias model directly from the target model, called GenB. In particular, GenB employs a generative network to learn the bias through a combination of the adversarial objective and knowledge distillation. We then debias our target model with GenB as a bias model, and show through extensive experiments the effects of our method on various VQA bias datasets including VQA-CP2, VQA-CP1, GQA-OOD, and VQA-CE.

Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition

“…Addressing the distribution shift is a crucial research problem since deep learning models are fragile to testing distribution different from the training [51]. In this aspect, various benchmarks have been proposed to measure the robustness under distribution shifts [9,14,23,25,26,29,30,45,48,50], and this problem has been extensively studied in broad research fields [3,4,10,15,16,24,38,39,40,43,52,55,62]. Among them, benchmarking robustness [23] and resolving scene bias [10,42] or distribution shift [43,59] are the most related to our problem setup.…”