Ranking comment sorting policies in online debates

Abstract

Online debates typically possess a large number of argumentative comments. Most readers who would like to see which comments are winning arguments often only read a part of the debate. Many platforms that host such debates allow for the comments to be sorted, say from the earliest to latest. How can argumentation theory be used to evaluate the effectiveness of such policies of sorting comments, in terms of the actually winning arguments displayed to a reader who may not have read the whole debate? We devise a pipeline that captures an online debate tree as a bipolar argumentation framework (BAF), which is sorted depending on the policy, giving a sequence of induced sub-BAFs representing how and how much of the debate has been read. Each sub-BAF has its own set of winning arguments, which can be quantitatively compared to the set of winning arguments of the whole BAF. We apply this pipeline to evaluate policies on Kialo debates, where it is shown that reading comments from most to least liked, on average, displays more winners than reading comments earliest first. Therefore, in Kialo, reading comments from most to least liked is on average more effective than reading from the earliest to the most recent.

GraphNLI: A Graph-based Natural Language Inference Model for Polarity Prediction in Online Debates

“…Understanding multiple viewpoints in online debates: A novel way to mitigate problems such as filter bubbles in large online debates is to present readers with a balanced sample of all the justified arguments representing the multiple important viewpoints. Once an argumentation framework has been induced from a discussion and the attack/support relation between posts has been established, it is possible to use the tools of argumentation theory to compute the "winning" arguments from all sides of a debate [41][42][43]: arguments which have not been refuted and are left standing as valid viewpoints (whether one may agree with them or not).…”

“…This could potentially be used to present only the justified arguments as summary to a reader. Previous work has also looked at how the conclusions of a logical reader can change depending on which parts of a debate they read, thus underscoring the dangers of sampling only parts of a large online debate [41][42][43]. Other work has shown how the location of justified arguments can be significantly influenced by whether the debate is acrimonious or supporting [10].…”

Salient deliberative norm types in comment sections on news sites

“…The salience of descriptive norms in comment sections is determined by technical affordances (Evans et al, 2017). An important technical affordance is the comment sorting (Young et al, 2021), because it determines the order of the comments. On news sites, comments are usually sorted chronologically (Strippel and Paasch-Colberg, 2020).…”

Argumentation frameworks with necessities and their relationship with logic programs

“…Several extensions and applications of bipolar AFs have been proposed in the literature. We can cite the use of bipolar AFs for text exploration [28], for detecting bipolar relations from texts [26], for supporting users [27] and ranking comment sorting policies [93] in inline debate and for social networks analysis [60].…”