Quarantining online hate speech: technical and ethical perspectives

Abstract

In this paper we explore quarantining as a more ethical method for delimiting the spread of Hate Speech via online social media platforms. Currently, companies like Facebook, Twitter, and Google generally respond reactively to such material: offensive messages that have already been posted are reviewed by human moderators if complaints from users are received. The offensive posts are only subsequently removed if the complaints are upheld; therefore, they still cause the recipients psychological harm. In addition, this approach has frequently been criticised for delimiting freedom of expression, since it requires the service providers to elaborate and implement censorship regimes. In the last few years, an emerging generation of automatic Hate Speech detection systems has started to offer new strategies for dealing with this particular kind of offensive online material. Anticipating the future efficacy of such systems, the present article advocates an approach to online Hate Speech detection that is analogous to the quarantining of malicious computer software. If a given post is automatically classified as being harmful in a reliable manner, then it can be temporarily quarantined, and the direct recipients can receive an alert, which protects them from the harmful content in the first instance. The quarantining framework is an example of more ethical online safety technology that can be extended to the handling of Hate Speech. Crucially, it provides flexible options for obtaining a more justifiable balance between freedom of expression and appropriate censorship.

AI-Assisted and Explainable Hate Speech Detection for Social Media Moderators – A Design Science Approach

“…Today, a large part of human communication takes place in the digital sphere, for instance via social media [1][2], and so does hate speech, which can be harmful for individuals and society as a whole [3]. Ullmann and Tomalin [4], for instance, describe that "[…] offensive posts are only subsequently removed if the complaints are upheld, therefore, they still cause the recipients psychological harm." (p. 1).…”

Developing an online hate classifier for multiple social media platforms

“…As an example, our classification model can be directly deployed to practical applications as well as further developed by other researchers. Regarding the use of the model in real systems (e.g., to automate moderation), we repeat the advice from a previous study [77] that essentially states that even small misclassification rates are a problem, as removing comments based on automatic detection methods can impact a user's freedom of speech in social media platforms [104]. It is highly unlikely that "perfect" classifiers for online hate would ever be developed, especially considering the subjective nature of what online hate is [72,75].…”

“…(p. 8). For these reasons, ethical considerations in online hate detection are important [104]. Therefore, we do not advocate letting the model automatically decide on banning or removal of messages (perhaps apart from situations where false positives play only a smaller role).…”

Dampak Ujaran Kebencian Di Media Sosial Pada Penggunaan Bahasa Indonesia Pada Warga Kepunduhan

“…In addition, this approach has frequently been criticised for delimiting freedom of expression, since it requires the service providers to elaborate and implement censorship regimes." [5]. Berbagai bentuk ungkapan yang menyakitkan dalam bentuk ekspresi maupun dengan kondisi yang lebih merugikan lagi.…”