Detection of Puffery on the English Wikipedia

Abstract

On Wikipedia, an online crowdsourced encyclopedia, volunteers enforce the encyclopedia's editorial policies. Wikipedia's policy on maintaining a neutral point of view has inspired recent research on bias detection, including "weasel words" and "hedges". Yet to date, little work has been done on identifying "puffery," phrases that are overly positive without a verifiable source. We demonstrate that collecting training data for this task requires some care, and construct a dataset by combining Wikipedia editorial annotations and information retrieval techniques. We compare several approaches to predicting puffery, and achieve 0.963 f1 score by incorporating citation features into a RoBERTa model. Finally, we demonstrate how to integrate our model with Wikipedia's public infrastructure to give back to the Wikipedia editor community.

WikiSQE: A Large-Scale Dataset for Sentence Quality Estimation in Wikipedia

“…The quality labels for detection were created by manually assigned to small-scale sentences (Herzig, Nunes, and Snir 2011;Hube and Fetahu 2019) or by using Wikipedia's Inline template. For the latter, Inline templates include "Citation needed" (Redi et al 2019), which indicates that a sentence needs citations, "Puffery" and "Peacock" (Bertsch and Bethard 2021), which indicates that the sentence contains exaggerated expressions, and the "Weasel words" (Ganter and Strube 2009), which indicates that the sentence contains ambiguous wording. A detailed comparison of our study with previous studies is given in (Ng et al 2014), linguistic acceptability (Warstadt, Singh, and Bowman 2019), automated essay estimation 4 , etc.…”

“…However, they are targeted at naive and superficial errors, while the remaining more multi-dimensional and in-depth evaluations are marked up by humans. Other work includes attempting to automatically detect specific labels given by editors (Ganter and Strube 2009;Redi et al 2019;Bertsch and Bethard 2021). The former is unable to make fine-grained evaluations of each sentence, while the latter focuses only on one particular poor-quality aspect.…”

Interpretable Classification of Wiki-Review Streams

“…According to the literature, the classification of wiki edits encompasses the detection of paid [20], puffery [21], reverted [22], [23], [24], toxic [25], [26] and vandal [9], [10], [12], [13], [17], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38] reviews. Similarly, prediction focuses on review quality [39], [40], [41] as well as on editor and article quality [18], [42], [43], [44], [45].…”