Learning End-to-End Goal-Oriented Dialog with Multiple Answers

Rajendran, Janarthanan; Ganhotra, Jatin; Singh, Satinder; Polymenakos, Lazaros

doi:10.18653/v1/d18-1418

Cited by 27 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3 Modified bAbI dialog tasks bAbI dialog tasks (referred to as original-bAbI dialog tasks from here on) were proposed by Bordes et al (2017) as a testbed to evaluate the strengths and shortcomings of end-to-end dialog systems in goal-oriented applications (Seo et al, 2017;Rajendran et al, 2018). The data set is generated by a restaurant reservation simulation where the Figure 2: Modified-bAbI dialog task.…”

Section: Proposed Methodsmentioning

confidence: 99%

Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use

Rajendran

Ganhotra²,

Polymenakos³

2019

Transactions of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Neural end-to-end goal-oriented dialog systems showed promise to reduce the workload of human agents for customer service, as well as reduce wait time for users. However, their inability to handle new user behavior at deployment has limited their usage in real world. In this work, we propose an end-to-end trainable method for neural goal-oriented dialog systems which handles new user behaviors at deployment by transferring the dialog to a human agent intelligently. The proposed method has three goals: 1) maximize user's task success by transferring to human agents, 2) minimize the load on the human agents by transferring to them only when it is essential and 3) learn online from the human agent's responses to reduce human agents load further. We evaluate our proposed method on a modified-bAbI dialog task that simulates the scenario of new user behaviors occurring at test time. Experimental results show that our proposed method is effective in achieving the desired goals.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use

Rajendran

Ganhotra²,

Polymenakos³

2019

Transactions of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Goal-oriented dialogue requires the agent to complete a related task with a clear goal through multiturn conversations. Although goal-oriented spoken and text-based dialogues have been studied in Natural Language Processing committee for years [4,15,24], goaloriented visual dialogue extends the setting to vision domain and is a relatively new and challenging field. Representatively, Guess-What?!…”

Section: Related Workmentioning

confidence: 99%

Answer-Driven Visual State Estimator for Goal-Oriented Visual Dialogue

Feng

Wang

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

A goal-oriented visual dialogue involves multi-turn interactions between two agents, Questioner and Oracle. During which, the answer given by Oracle is of great significance, as it provides golden response to what Questioner concerns. Based on the answer, Questioner updates its belief on target visual content and further raises another question. Notably, different answers drive into different visual beliefs and future questions. However, existing methods always indiscriminately encode answers after much longer questions, resulting in a weak utilization of answers. In this paper, we propose an Answer-Driven Visual State Estimator (ADVSE) to impose the effects of different answers on visual states. First, we propose an Answer-Driven Focusing Attention (ADFA) to capture the answerdriven effect on visual attention by sharpening question-related attention and adjusting it by answer-based logical operation at each turn. Then based on the focusing attention, we get the visual state estimation by Conditional Visual Information Fusion (CVIF), where overall information and difference information are fused conditioning on the question-answer state. We evaluate the proposed ADVSE to both question generator and guesser tasks on the large-scale GuessWhat?! dataset and achieve the state-of-the-art performances on both tasks. The qualitative results indicate that the ADVSE boosts the agent to generate highly efficient questions and obtains reliable visual attentions during the reasonable question generation and guess processes. CCS CONCEPTS • Computing methodologies → Computer vision tasks; Discourse, dialogue and pragmatics; Natural language generation; Computer vision representations.

show abstract

“…Most earlier works made use of a predefined template (Lemon et al, 2006;Wang and Lemon, 2013) to generate dialogues. More recently, deep neural networks have been used for building end-to-end architectures capable of generating questions (Vinyals and Le, 2015;Sordoni et al, 2015) and also for the task of goal-oriented dialogue generation (Rajendran et al, 2018;Bordes et al, 2017). Visual dialogue focuses on having a conversation about an image with either one or both of the agents being a machine.…”

Section: Dialogue Generation and Visual Dialoguementioning

confidence: 99%

What Should I Ask? Using Conversationally Informative Rewards for Goal-oriented Visual Dialog.

Shukla¹,

Elmadjian²,

Sharan³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

The ability to engage in goal-oriented conversations has allowed humans to gain knowledge, reduce uncertainty, and perform tasks more efficiently. Artificial agents, however, are still far behind humans in having goaldriven conversations. In this work, we focus on the task of goal-oriented visual dialogue, aiming to automatically generate a series of questions about an image with a single objective. This task is challenging, since these questions must not only be consistent with a strategy to achieve a goal, but also consider the contextual information in the image. We propose an end-to-end goal-oriented visual dialogue system, that combines reinforcement learning with regularized information gain. Unlike previous approaches that have been proposed for the task, our work is motivated by the Rational Speech Act framework, which models the process of human inquiry to reach a goal. We test the two versions of our model on the GuessWhat?! dataset, obtaining significant results that outperform the current state-of-the-art models in the task of generating questions to find an undisclosed object in an image.

show abstract

Learning End-to-End Goal-Oriented Dialog with Multiple Answers

Cited by 27 publications

References 20 publications

Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use

Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use

Answer-Driven Visual State Estimator for Goal-Oriented Visual Dialogue

What Should I Ask? Using Conversationally Informative Rewards for Goal-oriented Visual Dialog.

Contact Info

Product

Resources

About