WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Community

Abstract

We present a corpus that encompasses the complete history of conversations between contributors to Wikipedia, one of the largest online collaborative communities. By recording the intermediate states of conversationsincluding not only comments and replies, but also their modifications, deletions and restorations-this data offers an unprecedented view of online conversation. This level of detail supports new research questions pertaining to the process (and challenges) of large-scale online collaboration. We illustrate the corpus' potential with two case studies that highlight new perspectives on earlier work. First, we explore how a person's conversational behavior depends on how they relate to the discussion's venue. Second, we show that community moderation of toxic behavior happens at a higher rate than previously estimated. Finally the reconstruction framework is designed to be language agnostic, and we show that it can extract high quality conversational data in both Chinese and English.

Preemptive Toxic Language Detection in Wikipedia Comments Using Thread-Level Context

“…We found that on these examples the silver labels have 0.51 precision and 1.00 recall. This yields 0.67 F1 measure and is somewhat lower than the expected 0.85 obtained for this classifier in (Hua et al, 2018). The difference indicates that the thresholds from (Hua et al, 2018) obtained on non-deleted comments from Wikipedia may not perform equally well on deleted comments.…”

“…This yields 0.67 F1 measure and is somewhat lower than the expected 0.85 obtained for this classifier in (Hua et al, 2018). The difference indicates that the thresholds from (Hua et al, 2018) obtained on non-deleted comments from Wikipedia may not perform equally well on deleted comments. To address this and increase the quality of the labels, more deleted comments should be manually labeled and thresholds retuned using, e.g., the same error rate method of (Wulczyn et al, 2017a).…”

Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop

“…Zhang et al's 'Conversations Gone Awry' dataset consists of 1,270 conversations that took place between Wikipedia editors on publicly accessible talk pages. The conversations are sourced from the WikiConv dataset (Hua et al, 2018) and labeled by crowdworkers as either containing a personal attack from within (i.e., hostile behavior by one user in the conversation directed towards another) or remaining civil throughout.…”

Trajectories of Blocked Community Members: Redemption, Recidivism and Departure

“…In this work we use the complete conversational history between English Wikipedia editors on both article and user talk pages. With over 90 million conversations between 4 million users on 24 million talk pages, this is one of the largest collections of public conversations [20].…”