Constructive Language in News Comments

Abstract

We discuss the characteristics of constructive news comments, and present methods to identify them. First, we define the notion of constructiveness. Second, we annotate a corpus for constructiveness. Third, we explore whether available argumentation corpora can be useful to identify constructiveness in news comments. Our model trained on argumentation corpora achieves a top accuracy of 72.59% (baseline=49.44%) on our crowdannotated test data. Finally, we examine the relation between constructiveness and toxicity. In our crowd-annotated data, 21.42% of the non-constructive comments and 17.89% of the constructive comments are toxic, suggesting that non-constructive comments are not much more toxic than constructive comments.

Automatic Article Commenting: the Task and Dataset

“…Automatic article commenting poses new challenges due to the large input and output spaces and the open-domain nature of comments. Many efforts have been devoted to studying specific attributes of reader comments, such as constructiveness, persuasiveness, and sentiment Kolhatkar and Taboada, 2017;Barker et al, 2016). We introduce the new task of generating comments, and develop a dataset that is orders-of-magnitude larger than previous related corpus.…”

Dataset Creation for Ranking Constructive News Comments

“…In this paper, we directly evaluate the quality of comments separately from user feedback, focusing on their "constructiveness," as studied in (Napoles et al, 2017;Kolhatkar and Taboada, 2017). This quality measure is reasonable for services in that displaying constructive comments can stimulate discussion on a news article, which makes the user-generated content richer.…”

“…To avoid individual variation as much as possible, we need to prepare a more specific definition before annotation. We follow a previous study (Kolhatkar and Taboada, 2017) on constructiveness, where a questionnaire given to 100 people clarified detailed conditions for constructive comments. We digested it into several simple conditions, shown in Table 1, so that crowdsourced workers could systematically judge comments.…”

Unsupervised Ensemble of Ranking Models for News Comments Using Pseudo Answers

“…This line of research has included many studies on ranking comments according to user feedback [6,9,22]. On the other hand, there has also been much research on analyzing news comments in terms of "constructiveness" [7,13,18]. The most related research is Fujita et al [7].…”

“…However, this type of user-feedback is not suitable to assess the comment quality, because this type of measurement is biased by where a comment appears [7]; Earlier comments tend to receive more feedback since they will be displayed at the top of the page. In attempt of solving this problem, several studies introduce some aspects of the comment quality to focus on, e.g., constructiveness [7,13] or persuasiveness [22]. In particular, Fujita et al [7] proposed a new dataset to rank comments directly according to comment quality.…”