Multi‐document Summarization of Evaluative Text

Abstract

In many decision‐making scenarios, people can benefit from knowing what other people's opinions are. As more and more evaluative documents are posted on the Web, summarizing these useful resources becomes a critical task for many organizations and individuals. This paper presents a framework for summarizing a corpus of evaluative documents about a single entity by a natural language summary. We propose two summarizers: an extractive summarizer and an abstractive one. As an additional contribution, we show how our abstractive summarizer can be modified to generate summaries tailored to a model of the user preferences that is solidly grounded in decision theory and can be effectively elicited from users. We have tested our framework in three user studies. In the first one, we compared the two summarizers. They performed equally well relative to each other quantitatively, while significantly outperforming a baseline standard approach to multidocument summarization. Trends in the results as well as qualitative comments from participants suggest that the summarizers have different strengths and weaknesses. After this initial user study, we realized that the diversity of opinions expressed in the corpus (i.e., its controversiality) might play a critical role in comparing abstraction versus extraction. To clearly pinpoint the role of controversiality, we ran a second user study in which we controlled for the degree of controversiality of the corpora that were summarized for the participants. The outcome of this study indicates that for evaluative text abstraction tends to be more effective than extraction, particularly when the corpus is controversial. In the third user study we assessed the effectiveness of our user tailoring strategy. The results of this experiment confirm that user tailored summaries are more informative than untailored ones.

The Customer is Always Right: Sentiment Analysis for Bank Service Quality

“…Finally, we note that researchers have also studied the summarization of opinions in the tradition fashion, e.g., producing a short textual summary based on multiple reviews or even a single review [4,9,52,83,88]. Such a summary gives the reader a quick overview of what people think about a product or service.…”

A Qualitative Analysis of a Corpus of Opinion Summaries based on Aspects

“…For the annotation of our corpus, we randomly selected 10 reviews for each book of ReLi, taking as example other related works ( (Carenini et al, 2006), (Tadano et al, 2010)) that have used a similar number of opinions as data source. In the selection of reviews, we determined that they contain at most 300 words.…”

“…In Carenini et al (2006), 28 annotators created abstractive summaries for a corpus of reviews about a digital camera and a DVD player. Each participant in the annotation received 20 reviews randomly selected from the corpus and generated a summary of 100 words.…”

“…We choose the same number of words for these types of summaries to evaluate how they are generated under similar restrictions. A compression ratio in percentage (e.g., 25%) was not used because the vast majority of the works in aspect-based opinion summarization do not use this scheme (Carenini et al, 2006) (Ganesan et al, 2010) (Tadano et al, 2010).…”

Harvesting and Summarizing User-Generated Content for Advanced Speech-Based HCI

“…Summarization and opinion mining from user-generated content have been well studied for years, with many interesting derived topics [1], [2], [4], [6], [9], [10], [13], [20], [21], [26], [27], [28], [31], [32]. Summarization techniques, when applied to spoken dialogue systems, however, are much more complicated than those in pure-text systems.…”