Learning to Generate Reformulation Actions for Scalable Conversational Query Understanding

Xu, Zihan; Zhu, Jingtao; Geng, Ling; Yang, Yang; Lin, Bojia; Jiang, Daxin

doi:10.1145/3340531.3412112

Cited by 3 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Question completion. One line of work that suggests a more general approach to the ConvQA problem aims to create a selfcontained question from the incomplete utterance that can be answered by standalone QA systems [9,36,51,52,56]. Such approaches can either take the form of question rewriting, which generates a complete question from scratch [36,51], or question resolution, which adds relevant terms from the conversational history to the question [52].…”

Section: Related Workmentioning

confidence: 99%

Conversational Question Answering on Heterogeneous Sources

Christmann¹,

Roy²,

Weikum³

2022

Preprint

View full text Add to dashboard Cite

Conversational question answering (ConvQA) tackles sequential information needs where contexts in follow-up questions are left implicit. Current ConvQA systems operate over homogeneous sources of information: either a knowledge base (KB), or a text corpus, or a collection of tables. This paper addresses the novel issue of jointly tapping into all of these together, this way boosting answer coverage and confidence. We present Convinse, an end-to-end pipeline for ConvQA over heterogeneous sources, operating in three stages: i) learning an explicit structured representation of an incoming question and its conversational context, ii) harnessing this framelike representation to uniformly capture relevant evidences from KB, text, and tables, and iii) running a fusion-in-decoder model to generate the answer. We construct and release the first benchmark, ConvMix, for ConvQA over heterogeneous sources, comprising 3000 real-user conversations with 16000 questions, along with entity annotations, completed question utterances, and question paraphrases. Experiments demonstrate the viability and advantages of our method, compared to state-of-the-art baselines. CCS CONCEPTS• Information systems → Question answering.

show abstract

Section: Related Workmentioning

confidence: 99%

Conversational Question Answering on Heterogeneous Sources

Christmann¹,

Roy²,

Weikum³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Reformulations. In the parallel field on text-QA, question reformulation or rewriting has been pursued as conversational question completion [3,51,64,68,71]. In our work, we revive the more traditional sense of reformulations [10,13,25,28], where users pose queries in a different way when system responses are unsatisfactory.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs

Kaiser

Roy

Weikum

2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

The rise of personal assistants has made conversational question answering (ConvQA) a very popular mechanism for user-system interaction. State-of-the-art methods for ConvQA over knowledge graphs (KGs) can only learn from crisp question-answer pairs found in popular benchmarks. In reality, however, such training data is hard to come by: users would rarely mark answers explicitly as correct or wrong. In this work, we take a step towards a more natural learning paradigm -from noisy and implicit feedback via question reformulations. A reformulation is likely to be triggered by an incorrect system response, whereas a new follow-up question could be a positive signal on the previous turn's answer. We present a reinforcement learning model, termed Conqer, that can learn from a conversational stream of questions and reformulations. Conqer models the answering process as multiple agents walking in parallel on the KG, where the walks are determined by actions sampled using a policy network. This policy network takes the question along with the conversational context as inputs and is trained via noisy rewards obtained from the reformulation likelihood. To evaluate Conqer, we create and release ConvRef, a benchmark with about 11 natural conversations containing around 205 reformulations. Experiments show that Conqer successfully learns from noisy rewards, significantly improving over a state-of-the-art baseline.

show abstract

“…Reformulations. In the parallel field on text-QA, question reformulation or rewriting has been pursued as conversational question completion [3,53,67,71,74]. In our work, we revive the more traditional sense of reformulations [12,15,27,30], where users pose queries in a different way when system responses are unsatisfactory.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs

Kaiser,

Roy,

Weikum

2021

Preprint

View full text Add to dashboard Cite

The rise of personal assistants has made conversational question answering (ConvQA) a very popular mechanism for user-system interaction. State-of-the-art methods for ConvQA over knowledge graphs (KGs) can only learn from crisp question-answer pairs found in popular benchmarks. In reality, however, such training data is hard to come by: users would rarely mark answers explicitly as correct or wrong. In this work, we take a step towards a more natural learning paradigm -from noisy and implicit feedback via question reformulations. A reformulation is likely to be triggered by an incorrect system response, whereas a new follow-up question could be a positive signal on the previous turn's answer. We present a reinforcement learning model, termed Conqer, that can learn from a conversational stream of questions and reformulations. Conqer models the answering process as multiple agents walking in parallel on the KG, where the walks are determined by actions sampled using a policy network. This policy network takes the question along with the conversational context as inputs and is trained via noisy rewards obtained from the reformulation likelihood. To evaluate Conqer, we create and release ConvRef, a benchmark with about 11𝑘 natural conversations containing around 205𝑘 reformulations. Experiments show that Conqer successfully learns to answer conversational questions from noisy reward signals, significantly improving over a state-of-the-art baseline.

show abstract

Learning to Generate Reformulation Actions for Scalable Conversational Query Understanding

Cited by 3 publications

References 10 publications

Conversational Question Answering on Heterogeneous Sources

Conversational Question Answering on Heterogeneous Sources

Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs

Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs

Contact Info

Product

Resources

About