The SENSEI Annotated Corpus: Human Summaries of Reader Comment
            Conversations in On-line News

Barker, Emma; Paramita, Monica Lestari; Aker, Ahmet; Kurtić, Emina; Hepple, Mark; Gaizauskas, Robert

doi:10.18653/v1/w16-3605

Cited by 17 publications

(13 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automatic article commenting poses new challenges due to the large input and output spaces and the open-domain nature of comments. Many efforts have been devoted to studying specific attributes of reader comments, such as constructiveness, persuasiveness, and sentiment Kolhatkar and Taboada, 2017;Barker et al, 2016). We introduce the new task of generating comments, and develop a dataset that is orders-of-magnitude larger than previous related corpus.…”

Section: Related Workmentioning

confidence: 99%

Automatic Article Commenting: the Task and Dataset

Qin

Liu

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

Comments of online articles provide extended views and improve user engagement. Automatically making comments thus become a valuable functionality for online forums, intelligent chatbots, etc. This paper proposes the new task of automatic article commenting, and introduces a large-scale Chinese dataset 1 with millions of real comments and a humanannotated subset characterizing the comments' varying quality. Incorporating the human bias of comment quality, we further develop automatic metrics that generalize a broad set of popular reference-based metrics and exhibit greatly improved correlations with human evaluations.

show abstract

Section: Related Workmentioning

confidence: 99%

Automatic Article Commenting: the Task and Dataset

Qin

Liu

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

show abstract

“…Over the last decade, some research has been directed at the summarization of forum threads Hovy 2005, 2006;Tigelaar, op den Akker, and Hiemstra 2010). The majority of the recent work addressed summarization of comment threads on news websites (Ren et al 2011;Llewellyn, Grover, and Oberlander 2014;Giannakopoulos et al 2015;Kabadjov et al 2015;Aker et al 2016;Barker et al 2016). In the past decade, abstractive summarization techniques have been successfully applied to tasks related to discussion thread summarization, such as summarizing email threads (Zajic, Dorr, and Lin 2008), summarizing spoken and written conversations and meetings Oya et al 2014), and Twitter topic summarization (Zhang et al 2013).…”

Section: Summarization For Discussion Forum Threadsmentioning

confidence: 99%

Query-based summarization of discussion threads

et al. 2019

View full text Add to dashboard Cite

In this paper, we address query-based summarization of discussion threads. New users can profit from the information shared in the forum, Please check if the inserted city and country names in the affiliations are correct. if they can find back the previously posted information. However, discussion threads on a single topic can easily comprise dozens or hundreds of individual posts. Our aim is to summarize forum threads given real web search queries. We created a data set with search queries from a discussion forum’s search engine log and the discussion threads that were clicked by the user who entered the query. For 120 thread–query combinations, a reference summary was made by five different human raters. We compared two methods for automatic summarization of the threads: a query-independent method based on post features, and Maximum Marginal Relevance (MMR), a method that takes the query into account. We also compared four different word embeddings representations as alternative for standard word vectors in extractive summarization. We find (1) that the agreement between human summarizers does not improve when a query is provided that: (2) the query-independent post features as well as a centroid-based baseline outperform MMR by a large margin; (3) combining the post features with query similarity gives a small improvement over the use of post features alone; and (4) for the word embeddings, a match in domain appears to be more important than corpus size and dimensionality. However, the differences between the models were not reflected by differences in quality of the summaries created with help of these models. We conclude that query-based summarization with web queries is challenging because the queries are short, and a click on a result is not a direct indicator for the relevance of the result.

show abstract

“…e goal of DTS is to select the most relevant reply posts in a discussion thread and merge them to form a concise thread summary. Most of the current work focused on comment threads summarization on news websites [48][49][50][51][52][53]. Previous research studies have effectively attempted abstractive summarization (AS) techniques for the task of DTS, such as e-mail threads summarization [3], summarizing written and spoken chats/conversations and meeting recordings [4], and summarizing Twitter topic [5].…”

Section: Related Workmentioning

confidence: 99%

Sentence Embedding Based Semantic Clustering Approach for Discussion Thread Summarization

et al. 2020

View full text Add to dashboard Cite

Huge data on the web come from discussion forums, which contain millions of threads. Discussion threads are a valuable source of knowledge for Internet users, as they have information about numerous topics. The discussion thread related to single topic comprises a huge number of reply posts, which makes it hard for the forum users to scan all the replies and determine the most relevant replies in the thread. At the same time, it is also hard for the forum users to manually summarize the bulk of reply posts in order to get the gist of discussion thread. Thus, automatically extracting the most relevant replies from discussion thread and combining them to form a summary are a challenging task. With this motivation behind, this study has proposed a sentence embedding based clustering approach for discussion thread summarization. The proposed approach works in the following fashion: At first, word2vec model is employed to represent reply sentences in the discussion thread through sentence embeddings/sentence vectors. Next, K-medoid clustering algorithm is applied to group semantically similar reply sentences in order to reduce the overlapping reply sentences. Finally, different quality text features are utilized to rank the reply sentences in different clusters, and then the high-ranked reply sentences are picked out from all clusters to form the thread summary. Two standard forum datasets are used to assess the effectiveness of the suggested approach. Empirical results confirm that the proposed sentence based clustering approach performed superior in comparison to other summarization methods in the context of mean precision, recall, and F-measure.

show abstract

The SENSEI Annotated Corpus: Human Summaries of Reader Comment Conversations in On-line News

Cited by 17 publications

References 14 publications

Automatic Article Commenting: the Task and Dataset

Automatic Article Commenting: the Task and Dataset

Query-based summarization of discussion threads

Sentence Embedding Based Semantic Clustering Approach for Discussion Thread Summarization

Contact Info

Product

Resources

About