Extractive Opinion Summarization in Quantized Transformer Spaces

Abstract

We present the Quantized Transformer (QT), an unsupervised system for extractive opinion summarization. QT is inspired by Vector- Quantized Variational Autoencoders, which we repurpose for popularity-driven summarization. It uses a clustering interpretation of the quantized space and a novel extraction algorithm to discover popular opinions among hundreds of reviews, a significant step towards opinion summarization of practical scope. In addition, QT enables controllable summarization without further training, by utilizing properties of the quantized space to extract aspect-specific summaries. We also make publicly available Space, a large-scale evaluation benchmark for opinion summarizers, comprising general and aspect-specific summaries for 50 hotels. Experiments demonstrate the promise of our approach, which is validated by human studies where judges showed clear preference for our method over competitive baselines.

Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance

“…Recently, specifically for opinionated texts, several abstractive multi-document summarization methods have been developed, such as MeanSum, Copycat, and DenoiseSum, as explained in Section 4.3. Concurrently with our work, Angelidis et al (2021) use quantitized transformers enabling aspect-based extractive summarization, and Amplayo et al (2020) incorporate the aspect and sentiment distributions into the unsupervised abstractive summarization. Our method incorporates topic-tree structure into unsupervised abstractive summarization and generates summaries consisting of multiple granularities of topics.…”

StreamHover: Livestream Transcript Summarization and Annotation

“…The problem could be exacerbated by ungrammatical spoken utterances and transcription errors. Instead, we consider VQ-VAE, an unsupervised representation learning technique (van den Oord et al, 2017;Jin et al, 2020;Angelidis et al, 2021) for content extraction. Unsupervised training of the VQ-VAE model and its inference could potentially be performed at the same time, allowing important utterances to be extracted from a transcript segment on-the-fly during streaming, without interrupting the learning process.…”

“…The method was explored for opinion summarization (Angelidis et al, 2021) and machine translation (Prato et al, 2020). We are interested in using the method to account for domain characteristics of livestreams, which showcase new and creative work of artists and designers on their use of Photoshop, Illustrator, and other tools.…”

“…The summary selection process is deterministic and our encoder takes full advantage of a large, pre-trained model to produce initial utterance representations. This design sets our method apart from that of Angelidis et al (2021). Moreover, the method has the potential for modelling topic transitions between utterances to improve summarization of livestreams, which we leave for future work.…”

“…We make use of vector-quantized variational autoencoders (VQ-VAE; van den Oord et al, 2017) to identify salient utterances. The model has been applied successfully to opinion summarization that learns in-domain sentence representations (Angelidis et al, 2021), which is essential for adaptation of general-domain models. We refrain from using sequential methods for utterance selection.…”