Unsupervised Opinion Summarization as Copycat-Review Generation

Abstract

Opinion summarization is the task of automatically creating summaries that reflect subjective information expressed in multiple documents, such as product reviews. While the majority of previous work has focused on the extractive setting, i.e., selecting fragments from input reviews to produce a summary, we let the model generate novel sentences and hence produce abstractive summaries. Recent progress in summarization has seen the development of supervised models which rely on large quantities of document-summary pairs. Since such training data is expensive to acquire, we instead consider the unsupervised setting, in other words, we do not use any summaries in training. We define a generative model for a review collection which capitalizes on the intuition that when generating a new review given a set of other reviews of a product, we should be able to control the "amount of novelty" going into the new review or, equivalently, vary the extent to which it deviates from the input. At test time, when generating summaries, we force the novelty to be minimal, and produce a text reflecting consensus opinions. We capture this intuition by defining a hierarchical variational autoencoder model. Both individual reviews and the products they correspond to are associated with stochastic latent codes, and the review generator ("decoder") has direct access to the text of input reviews through the pointergenerator mechanism. Experiments on Amazon and Yelp datasets, show that setting at test time the review's latent code to its mean, allows the model to produce fluent and coherent summaries reflecting common opinions.

Unsupervised Opinion Summarization with Noising and Denoising

“…Unsupervised text generation methods (Freitag and Roy 2018;Fevry and Phang 2018;Chu and Liu 2019) conventionally make use of variational autoencoders (Kingma and Welling 2014), while employing relatively simple decoders in order to mitigate posterior collapse (Kingma and Welling 2014;Bowman et al 2016). A more recent line of work (Bražinskas, Lapata, and Titov 2019;Amplayo and Lapata 2020) creates synthetic datasets in cases where gold standard summaries are not available which in turn allow to train models in a supervised setting and make use of of effective decoding techniques such as attention and copy. Our method is in line with this work, but ultimately different in its use of content planning to guide both summarization and synthetic data creation.…”

“…The fused encodings are then passed to a decoder that uses the mean aspect and sentiment distributions as a content plan to generate output summary y. We do not employ an encoder in our model, but rather reuse the encodings from the content plan induction model, which improves memory-efficiency in comparison to related architectures (Chu and Liu 2019;Bražinskas, Lapata, and Titov 2019;Amplayo and Lapata 2020). At test time, the same model is used to summarize actual reviews.…”

“…Experiments on three datasets (Wang and Ling 2016;Chu and Liu 2019;Bražinskas, Lapata, and Titov 2019) representing different domains (movies, business, and product reviews) and summarization requirements (short vs longer summaries) show that our approach outperforms competitive systems in terms of ROUGE, achieving state of the art across the board. Human evaluation further confirms that the summaries produced by our model capture salient opinions as well as being coherent and fluent.…”

“…They consider an unsupervised learning setting where there are only documents (product or business reviews) available without corresponding summaries. An intuitive solution to the lack of training data is to create synthetic summary-review pairs (Amplayo and Lapata 2020;Bražinskas, Lapata, and Titov 2019) by sampling a review from a corpus of product reviews, and pretending it is a summary.…”

“…Although synthetic datasets enable the use of supervised training and have been found to produce higher quality summaries than autoencoder-based methods (Chu and Liu 2019), they cannot, by definition, resemble real-world data. Bražinskas, Lapata, and Titov (2019) rely on random sam-pling to select the pseudo-summary which might have no connection to the input it purports to summarize. Amplayo and Lapata (2020) create multiple input reviews by adding noise to the sampled summary.…”