Learning Goal-Oriented Visual Dialog Agents: Imitating and Surpassing Analytic Experts

Abstract

This paper tackles the problem of learning a questioner in the goal-oriented visual dialog task. Several previous works adopt model-free reinforcement learning. Most pretrain the model from a finite set of human-generated data. We argue that using limited demonstrations to kick-start the questioner is insufficient due to the large policy search space. Inspired by a recently proposed information theoretic approach, we develop two analytic experts to serve as a source of highquality demonstrations for imitation learning. We then take advantage of reinforcement learning to refine the model towards the goal-oriented objective. Experimental results on the GuessWhat?! dataset show that our method has the combined merits of imitation and reinforcement learning, achieving the state-of-the-art performance.

Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review

“…The History-Advantage Sequence Training (HAST) comprehensively integrates the dialogue history with co-attention modules for visual context in history encoding and one history-aware gate. For more efficient training dialogue models with RL, the probabilistic framework is introduced [9] with an Information Gain Expert and a Target Posterior Expert, which provide virtually unlimited expert demonstrations for pre-training the questioner and will be refined for a even better policy with RL.…”

SKANet: Structured Knowledge-Aware Network for Visual Dialog

“…Currently, bridging the gap between vision and language, such as visual captioning [1,2], visual question answering [3,4] and visual dialog [5,6,7], has attracted huge attention in the computer vision community. Visual dialog can be regarded as an extension of VQA.…”