Modeling Ambiguity, Subjectivity, and Diverging Viewpoints in Opinion Question Answering Systems

Abstract

Abstract-Product review websites provide an incredible lens into the wide variety of opinions and experiences of different people, and play a critical role in helping users discover products that match their personal needs and preferences. To help address questions that can't easily be answered by reading others' reviews, some review websites also allow users to pose questions to the community via a question-answering (QA) system. As one would expect, just as opinions diverge among different reviewers, answers to such questions may also be subjective, opinionated, and divergent. This means that answering such questions automatically is quite different from traditional QA tasks, where it is assumed that a single 'correct' answer is available. While recent work introduced the idea of question-answering using product reviews, it did not account for two aspects that we consider in this paper: (1) Questions have multiple, often divergent, answers, and this full spectrum of answers should somehow be used to train the system; and (2) What makes a 'good' answer depends on the asker and the answerer, and these factors should be incorporated in order for the system to be more personalized. Here we build a new QA dataset with 800 thousand questions-and over 3.1 million answers-and show that explicitly accounting for personalization and ambiguity leads both to quantitatively better answers, but also a more nuanced view of the range of supporting, but subjective, opinions.

Why Does a Visual Question Have Different Answers?

“…The closely-following second and third most common reasons are answer granularity (GRN) and synonyms (SYN) which account for 72.9% and 68.3% of VQAs across both datasets ( Figure 3c; 2 person threshold). These findings highlight that most answer differences can be resolved by disambiguating visual questions or resolving synonyms and differing granularity [16,27,43].…”

“…We offer our work as a valuable foundation for improving VQA services, by empowering system designers and users to know how to prevent, interpret, or resolve answer differences. Specifically, a solution that anticipates why a visual question will lead to different answers (summarized in Figure 1) could (1) help users identify how to modify their visual question in order to arrive at a single, unambiguous answer; e.g., retake an image when it is low quality or does not show the answer versus modify the question when it is ambiguous or invalid; (2) increase users' awareness for what reasons, if any, trigger answer differences when they are given a single answer; or (3) reveal how to automatically aggregate different answers [2,19,24,26,43] when multiple answers are collected.…”

Training Neural Response Selection for Task-Oriented Dialogue Systems

“…All datasets are in the form of (input, response) pairs. For UBUNTU 8 , SEMEVAL15 9 , and AMAZONQA 10 we use standard data splits into training, dev, and test portions following the original work (Lowe et al, 2017;Nakov et al, 2015;Wan and McAuley, 2016). For the OpenSubtitles dataset (OPENSUB) (Lison and Tiedemann, 2016), we rely on the data splits introduced by Henderson et al (2019).…”

Review-guided Helpful Answer Identification in E-commerce

“…The statistics are shown in Table 1. The original question-answer pairs are from a public data collection crawled by Wan and McAuley [34]. We also utilize the product ID in the QA dataset to align with the reviews in Amazon review dataset [14] so that the corresponding reviews of each product can be obtained.…”