Content moderation, AI, and the question of scale

Gillespie, Tarleton

doi:10.1177/2053951720943234

Cited by 238 publications

(166 citation statements)

References 16 publications

Supporting

Mentioning

160

Contrasting

Unclassified

Order By: Relevance

“…Our results shed light into this new genre, which is beginning to be explored not just from a linguistic point of view, but also from the point of view of content moderation (Gillespie, 2018(Gillespie, , 2020Risch & Krestel, 2018;Seering, Wang, Yoon & Kaufman 2019).…”

Section: Discussionmentioning

confidence: 86%

A corpus analysis of online news comments using the Appraisal framework

Cavasso

Taboada

2021

Journal of Corpora and Discourse Studies

View full text Add to dashboard Cite

We present detailed analyses of the distribution of Appraisal categories (Martin & White, 2005) in a corpus of online news comments. The corpus consists of just over one thousand comments posted in response to a variety of opinion pieces on the website of the Canadian English-language newspaper The Globe and Mail. We annotated all the comments with labels corresponding to different categories of the Appraisal framework. Analyses of the annotations show that comments are overwhelmingly negative and that they favour two of the subtypes of Attitude, Judgement and Appreciation. The paper contributes a methodology for annotating Appraisal, examining the interaction of Appraisal with negation, the constructive nature of comments, and the level of toxicity found in them. The results show that highly opinionated language is expressed as opinion (Judgement and Appreciation) rather than as an emotional reaction (Affect). This finding, together with the interplay of evaluative language with constructiveness and toxicity in the comments, can be applied to the automatic moderation of online comments.

show abstract

Section: Discussionmentioning

confidence: 86%

A corpus analysis of online news comments using the Appraisal framework

Cavasso

Taboada

2021

Journal of Corpora and Discourse Studies

View full text Add to dashboard Cite

show abstract

“…Based on his recent research into AI data workers in South America, Schmidt (2019) has pointed out that most of the value is in validation data, where ‘human cognition is needed to evaluate the decisions that machine-learning systems have made’ (9). Moreover, research has pointed out how Facebook obscures its human content moderators (Gillespie, 2020), and how Google uses human raters to perform quality assessments (Bilić, 2016). The type of validation work is also found in the use of human labor in home assistants, such as Amazon Echo and Google Home (Day et al., 2019; Verheyden et al., 2019).…”

Section: The Invisible Backstage Of Ai Productionmentioning

confidence: 99%

Lifting the curtain: Strategic visibility of human labour in AI-as-a-Service

Newlands

2021

Big Data & Society

View full text Add to dashboard Cite

Artificial Intelligence-as-a-Service (AIaaS) empowers individuals and organisations to access AI on-demand, in either tailored or ‘off-the-shelf’ forms. However, institutional separation between development, training and deployment can lead to critical opacities, such as obscuring the level of human effort necessary to produce and train AI services. Information about how, where, and for whom AI services have been produced are valuable secrets, which vendors strategically disclose to clients depending on commercial interests. This article provides a critical analysis of how AIaaS vendors manipulate the visibility of human labour in AI production based on whether the vendor relies on paid or unpaid labour to fill interstitial gaps. Where vendors are able to occlude human labour in the organisational ‘backstage,’ such as in data preparation, validation or impersonation, they do so regularly, further contributing to ongoing techno-utopian narratives of AI hype. Yet, when vendors must co-produce the AI service with the client, such as through localised AI training, they must ‘lift the curtain’, resulting in a paradoxical situation of needing to both perpetuate dominant AI hype narratives while emphasising AI’s mundane limitations.

show abstract

“…The expanding volume and velocity of user participation makes it increasingly difficult and expensive to rapidly detect and remove undesirable posts (Sood et al, 2012;Gillespie, 2020). Fully automated Machine Learning (ML) approaches for text classifications have shown remarkable improvements over the last years.…”

Section: Introductionmentioning

confidence: 99%

“…Fully automated Machine Learning (ML) approaches for text classifications have shown remarkable improvements over the last years. However, ML models still lack user acceptance and applicability (Brunk et al, 2019;Gillespie, 2020). Fully automated approaches are known to be error-prone (Scharkow, 2013) and rarely reach the level of accuracy required to be applied in real-word settings.…”

Section: Introductionmentioning

confidence: 99%

REM: Efficient Semi-Automated Real-Time Moderation of Online Forums

Andersen¹,

Zukunft²,

Maalej³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

This paper presents REM, a novel tool for the semi-automated real-time moderation of large scale online forums. The growing demand for online participation and the increasing number of user comments raise challenges in filtering out harmful and undesirable content from public debates in online forums. Since a manual moderation does not scale well and pure automated approaches often lack the required level of accuracy, we suggest a semi-automated moderation approach. Our approach maximizes the efficiency of manual efforts by targeting only those comments for which human intervention is needed, e.g. due to high classification uncertainty. Our tool offers a rich visual interactive environment enabling the exploration of online debates. We conduct a preliminary evaluation experiment to demonstrate the suitability of our approach and publicly release the source code of REM.

show abstract

Content moderation, AI, and the question of scale

Cited by 238 publications

References 16 publications

A corpus analysis of online news comments using the Appraisal framework

A corpus analysis of online news comments using the Appraisal framework

Lifting the curtain: Strategic visibility of human labour in AI-as-a-Service

REM: Efficient Semi-Automated Real-Time Moderation of Online Forums

Contact Info

Product

Resources

About