Resources and benchmark corpora for hate speech detection: a systematic review

Abstract

Hate Speech in social media is a complex phenomenon, whose detection has recently gained significant traction in the Natural Language Processing community, as attested by several recent review works. Annotated corpora and benchmarks are key resources, considering the vast number of supervised approaches that have been proposed. Lexica play an important role as well for the development of hate speech detection systems. In this review, we systematically analyze the resources made available by the community at large, including their development methodology, topical focus, language coverage, and other factors. The results of our analysis highlight a heterogeneous, growing landscape, marked by several issues and venues for improvement.

“Contro L’Odio”: A Platform for Detecting, Monitoring and Visualizing Hate Speech against Immigrants in Italian Social Media

“…In the last few years several works contributed to the development of HS detection automatic methods, both releasing novel annotated resources, lexicons of hate words or presenting automated classifiers. Two surveys (Schmidt and Wiegand 2017;Fortuna and Nunes 2018) and a systematic review were recently published on this topic (Poletto et al 2020). For what concerns Italian, a few resources have been recently developed using data from Twitter (Sanguinetti et al 2018;Poletto et al 2017;Patti 2019), Facebook (Del Vigna et al 2017) and Instagam (Corazza et al 2019).…”

“…For a more complete overview of the available HS resources, including lexica and benchmark datasets, in Italian and in other languages, we refer to Poletto et al (2020).…”

“…It is worth pointing out that the implementation of effective monitoring tools cannot disregard the problem of defining what hate speech is. This issue, as already emerged in Poletto et al (2020), is strongly posed as a question to be addressed. Finding a univocal and satisfactory definition of the hate speech phenomenon is challenging, and one of the main difficulties lies in drawing boundaries between hate speech and other broader phenomena such as offensive language.…”

“…Second, the choice of such general keywords for data gathering risks not to be effective in monitoring the evolution of the debate around vulnerable minorities. However, although the drawbacks of keyword-based approaches are known to researchers (Davidson, Bhattacharya, and Weber 2019), there are currently no clear alternatives to this technique, as discussed in Poletto et al (2020).…”

A Systematic Literature Review of Different Machine Learning Methods on Hate Speech Detection

“…Other systematic literature review such as [6] use soft computing techniques and [7] use benchmark corpora. In this research, we focus on using the word "hate speech" which according to [8] is a term that hits a particular community or individual that makes them suffer, while the opposition doesn't care.…”