Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages

Abstract

Abstractive summarization is a technique that allows for extracting condensed meanings from long texts, with a variety of potential practical applications. Nonetheless, today’s abstractive summarization research is limited to testing the models on various types of data, which brings only marginal improvements and does not lead to massive practical employment of the method. In particular, abstractive summarization is not used for social media research, where it would be very useful for opinion and topic mining due to the complications that social media data create for other methods of textual analysis. Of all social media, Reddit is most frequently used for testing new neural models of text summarization on large-scale datasets in English, without further testing on real-world smaller-size data in various languages or from various other platforms. Moreover, for social media, summarizing pools of texts (one-author posts, comment threads, discussion cascades, etc.) may bring crucial results relevant for social studies, which have not yet been tested. However, the existing methods of abstractive summarization are not fine-tuned for social media data and have next-to-never been applied to data from platforms beyond Reddit, nor for comments or non-English user texts. We address these research gaps by fine-tuning the newest Transformer-based neural network models LongFormer and T5 and testing them against BART, and on real-world data from Reddit, with improvements of up to 2%. Then, we apply the best model (fine-tuned T5) to pools of comments from Reddit and assess the similarity of post and comment summarizations. Further, to overcome the 500-token limitation of T5 for analyzing social media pools that are usually bigger, we apply LongFormer Large and T5 Large to pools of tweets from a large-scale discussion on the Charlie Hebdo massacre in three languages and prove that pool summarizations may be used for detecting micro-shifts in agendas of networked discussions. Our results show, however, that additional learning is definitely needed for German and French, as the results for these languages are non-satisfactory, and more fine-tuning is needed even in English for Twitter data. Thus, we show that a ‘one-for-all’ neural-network summarization model is still impossible to reach, while fine-tuning for platform affordances works well. We also show that fine-tuned T5 works best for small-scale social media data, but LongFormer is helpful for larger-scale pool summarizations.

Social Media Text Summarization: A Survey Towards a Transformer-based System Design

“…It is important to emphasize that, in the field of social media summarization, few research works [7,8,9] have demonstrated transformer-based social media abstractive text summarization systems. Our system aims to optimize the results achieved so far in two main points.…”

“…A Transformer-based abstractive summarization is presented in [7] which creates summaries in three languages from posts and comment pools. The datasets are from Reddit and Twitter.…”

Enhancing Text Summarization with a T5 Model and Bayesian Optimization

“…In carrying out text summary tasks there are many techniques that can be used, one of which is using the most popular models such as the Transformer-based Text-to-Text Transfer Transformer or the T5 model which has advantages in producing good text summaries. However, several studies conducted by Mastropaolo et al [21], Fendji et al [22], and Blekanov et al [23] revealed that the T5 model still has room to improve performance in the process of carrying out text summary tasks, this study will use Bayesian optimization techniques that will perform a task in increasing the performance of the T5 model. In the Bayesian optimization task, we will use Bayesian probability theory for an iterative model so that it can have the advantage of updating initial knowledge.…”

Editorial for the Special Issue “Selected Papers from the 9th Annual Conference ‘Comparative Media Studies in Today’s World’ (CMSTW’2021)”

“…Everyday communication similar to that described in [12] tends to create communitiesan element of communication architecture with dynamic composition, blurred borders, thresholds to enter, and internal "center-periphery" structure [13]. Detecting communities within socially mediated discussions helps to rethink the structure of social grouping as well as its reasons.…”

“…Trying to answer the questions posed above, the papers link the phenomena of media, psychological, or social-grouping nature to platform affordances, hierarchies of communicators, geographical proximity of communication loci, or dynamics of seemingly structureless opinion cumulation [11]. Thus, [12] demonstrates, among others, that large-scale international discussions on social networks, such as Twitter, have a certain logic in how their news-like content changes into discussion on problematic issues with varying speeds, depending on the distance from the geographical epicenter of the discussed event. With predominantly methodological goals, the authors fine-tune and test several neural-networkbased models for abstractive summarization, but they also show how summarization applied to real-world online debates allows for discovering their news/issues structure from a comparative perspective.…”