Detecting Wikipedia vandalism via spatio-temporal analysis of revision metadata?

Abstract

Blatantly unproductive edits undermine the quality of the collaboratively-edited encyclopedia, Wikipedia. They not only disseminate dishonest and offensive content, but force editors to waste time undoing such acts of vandalism. Language-processing has been applied to combat these malicious edits, but as with email spam, these filters are evadable and computationally complex. Meanwhile, recent research has shown spatial and temporal features effective in mitigating email spam, while being lightweight and robust.In this paper, we leverage the spatio-temporal properties of revision metadata to detect vandalism on Wikipedia. An administrative form of reversion called rollback enables the tagging of malicious edits, which are contrasted with nonoffending edits in numerous dimensions. Crucially, none of these features require inspection of the article or revision text. Ultimately, a classifier is produced which flags vandalism at performance comparable to the natural-language efforts we intend to complement (85% accuracy at 50% recall). The classifier is scalable (processing 100+ edits a second) and has been used to locate over 5,000 manually-confirmed incidents of vandalism outside our labeled set. KeywordsDesign, Measurement, Performance, Security, Wikipedia, spatio-temporal reputation, vandalism, collaborative software, content-based access control ABSTRACTBlatantly unproductive edits undermine the quality of the collaboratively-edited encyclopedia, Wikipedia. They not only disseminate dishonest and offensive content, but force editors to waste time undoing such acts of vandalism. Language-processing has been applied to combat these malicious edits, but as with email spam, these filters are evadable and computationally complex. Meanwhile, recent research has shown spatial and temporal features effective in mitigating email spam, while being lightweight and robust.In this paper, we leverage the spatio-temporal properties of revision metadata to detect vandalism on Wikipedia. An administrative form of reversion called rollback enables the tagging of malicious edits, which are contrasted with nonoffending edits in numerous dimensions. Crucially, none of these features require inspection of the article or revision text. Ultimately, a classifier is produced which flags vandalism at performance comparable to the natural-language efforts we intend to complement (85% accuracy at 50% recall). The classifier is scalable (processing 100+ edits a second) and has been used to locate over 5,000 manually-confirmed incidents of vandalism outside our labeled set.

Trust in collaborative web applications

“…The notion of author reputation was also investigated by West et al [65]. Rather than doing fine-grained content analysis of Adler, West detects an administrative form of revert called rollback to negatively impact the reputations of offending editors.…”

“…Meanwhile, inspired by the use of metadata to combat email spam [36], West et al [65] concentrate on a set of content-exclusive metadata features based on spatio-temporal properties. Simple properties include the time when an edit was made, the length of the revision comment, etc.. More novel are reputations generated from metadata-driven detection of revert actions.…”

“…Nearly 80% of vandalism is committed by anonymous users [65], who contribute only 31% of all article edits [15]. Goldman [35] notes that anonymous users are sometimes treated as "second class citizens" and that their edits undergo additional scrutiny.…”

“…For example, the WikiScanner tool [14] detects conflicts-of-interest based on IP geo-location (e.g., Edits from an IP from Redmond, Washington to the "Microsoft" article might warrant extra scrutiny). Similarly, [65] computes geographical reputations based on geo-location that prove effective in predicting the behavior of new users. Such analysis is not possible when IP addresses are not available.…”

“…Certainly, systems that arrive at user-reputations associatively (citation-based, NLP-based) are less affected than those that compute user reputations directly (content-driven, metadata-based). For the latter class, it is important that mechanisms are in place to evaluate users in the absence of history (for example, the spatial reputations of [65]). Secondly, if trust values are used to incentivize good behavior (see Sec.…”