Making History Matter: History-Advantage Sequence Training for Visual Dialog

Abstract

We study the multi-round response generation in visual dialog, where a response is generated according to a visually grounded conversational history. Given a triplet: an image, Q&A history, and current question, all the prevailing methods follow a codec (i.e., encoder-decoder) fashion in a supervised learning paradigm: a multimodal encoder encodes the triplet into a feature vector, which is then fed into the decoder for the current answer generation, supervised by the ground-truth. However, this conventional supervised learning does NOT take into account the impact of imperfect history, violating the conversational nature of visual dialog and thus making the codec more inclined to learn history bias but not contextual reasoning. To this end, inspired by the actor-critic policy gradient in reinforcement learning, we propose a novel training paradigm called History-Advantage Sequence Training (HAST). Specifically, we intentionally impose wrong answers in the history, obtaining an adverse critic, and see how the historic error impacts the codec's future behavior by History Advantagea quantity obtained by subtracting the adverse critic from the gold reward of ground-truth history. Moreover, to make the codec more sensitive to the history, we propose a novel attention network called History-Aware Co-Attention Network (HACAN) which can be effectively trained by using HAST. Experimental results on three benchmarks: VisDial v0.9&v1.0 and GuessWhat?!, show that the proposed HAST strategy consistently outperforms the state-of-the-art supervised counterparts.

NMN-VD: A Neural Module Network for Visual Dialog

“…Das et al [ 2 ] and Lu et al [ 7 ] adopted a dialog history attention mechanism to find and focus on past dialog history related to the present question. Wu et al [ 10 ], Guo et al [ 4 ], and Yang et al [ 11 ] proposed a model for applying the co-attention mechanism among the three elements of current question, image, and past dialog history to determine the answer to the current question. Gan et al [ 3 ] proposed a model that repeats co-attention among the three elements several times.…”

“…The existing models for visual dialog have been mostly implemented with a large monolithic neural network [ 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 ]. However, VQA and visual dialog are composable in nature in that the process of generating an answer to one natural language question can be completed by composing multiple basic neural network modules.…”

Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog

“…We also include some concurrent work for visual dialog that has not been discussed above, including image-questionanswer synergistic network (Guo et al, 2019), recursive visual attention (Niu et al, 2018), factor graph attention (Schwartz et al, 2019), dual attention network (Kang et al, 2019), graph neural network , history-advantage sequence training (Yang et al, 2019), and weighted likelihood estimation .…”

Answer-Driven Visual State Estimator for Goal-Oriented Visual Dialogue

“…Guesser model is evaluated by classification error rate. The 2 baseline models [6]: HRED, HRED-VGG, 3 attention-based models PLAN [28], A-ATT [7], HACAN [25], and 2 Feature-wise Linear Modulation (FiLM) models: single-hop FiLM [14], multi-hop FiLM [23], are compared. Table 3 compares the test error of Guess models.…”