The COVID-19 Mental Health Content Moderation Conundrum

Abstract

At the time of writing (mid-May 2020), mental health charities around the world have experienced an unprecedented surge in demand. At the same time, record-high numbers of people are turning to social media to maintain personal connections due to restrictions on physical movement. But organizations like the mental health charity Mind and even the UK Government have expressed concerns about the possible strain on mental health that may come from spending more time online during COVID-19. These concerns are unsurprising, as debates about the link between heavy social media use and mental illness raged long before the pandemic. But our newly heightened reliance on platforms to replace face-to-face communication has created even more pressure for social media companies to heighten their safety measures and protect their most vulnerable users. To develop and enact these changes, social media companies are reliant on their content moderation workforces, but the COVID-19 pandemic has presented them with two related conundrums: (1) recent changes to content moderation workforces means platforms are likely to be less safe than they were before the pandemic and (2) some of the policies designed to make social media platforms safer for people’s mental health are no longer possible to enforce. This Social Media + Society: 2K essay will address these two challenges in depth.

Examining embedded apparatuses of AI in Facebook and TikTok

“…Moreover, it is not only adults, but in particular younger individuals that are more likely to get their news information from social media platforms like TikTok (Swart 2021 ). On social media generally and TikTok specifically, scholars have called for greater moderation of problematic mental health representations and content (Gerrard 2020 ) and the spread of extremist content and hateful speech (Weimann and Masri 2020 ). These concerns are joined by examinations of algorithmic bias when it comes to categories of race, class, and gender (Bishop 2021 ), as well as how TikTok contributes to images of postfeminist neoliberal ideals of girlhood through promotion of a narrow range of prominent influencers (Kennedy 2020 ).…”

Envisioning Post-pandemic Digital Neurological, Psychiatric and Mental Health Care

“…Mental health charities across the globe experienced substantial increase in demand over the course of the pandemic (31). These trusted and crucial components of the care ecosystem have also highlighted the potential links between mental health strain and increased online activity during the pandemic, and the problems in effectively policing content shared on social media by peers (32). At the time of writing the pandemic shows only moderate signs of abatement globally, following which we can expect publication of a large corpus of clinical studies which will provide statistically rigid data to inform post pandemic deployment of telehealth in managed care.…”

Acting like a bot as a defiance of platform power: Examining YouTubers’ patterns of ‘inauthentic’ behaviour on Twitter during COVID-19

“…information disorders and lockdowns) that are generative of unforeseen connections among actors, themes, and objects in public ‘debates’ (Callon et al, 2011). One of these unforeseen connections is how COVID-19 pushed social media platforms to take unprecedented (mostly automated) steps to moderate content (Gerrard, 2020; Vijaya and Derella, 2020) and to develop new policies on ‘appropriate’ communication (e.g. Krishnan et al, 2021), which we argue may have an impact on emerging social media content creators’ attempts to gain visibility online.…”