Does BERT Pay Attention to Cyberbullying?

Elsafoury, Fatma; Katsigiannis, Stamos; Wilson, Steven R.; Ramzan, Naeem

doi:10.1145/3404835.3463029

Cited by 15 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I also investigate the most important part of speech (POS) tags that BERT relies on for its performance. The results of this work suggest that pre-training BERT results in a syntactical bias that impacts its performance on the task of hate speech detection (Elsafoury et al, 2021b).…”

Section: The Explainability Perspectivementioning

confidence: 89%

Darkness can not drive out darkness: Investigating Bias in Hate SpeechDetection Models

Elsafoury¹

2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

View full text Add to dashboard Cite

This paper is a summary of the work in my PhD thesis. In which, I investigate the impact of bias in NLP models on the task of hate speech detection from three perspectives: explainability, offensive stereotyping bias, and fairness. I discuss the main takeaways from my thesis and how they can benefit the broader NLP community. Finally, I discuss important future research directions. The findings of my thesis suggest that bias in NLP models impacts the task of hate speech detection from all three perspectives. And that unless we start incorporating social sciences in studying bias in NLP models, we will not effectively overcome the current limitations of measuring and mitigating bias in NLP models.

show abstract

Section: The Explainability Perspectivementioning

confidence: 89%

Darkness can not drive out darkness: Investigating Bias in Hate SpeechDetection Models

Elsafoury¹

2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

View full text Add to dashboard Cite

show abstract

“…The attention mechanism on which it is based makes it possible for it to handle long-term dependencies [16,17]. Hence, Transformer-based models have gained increased attention in HS detection and classification [12,15,18,19]. Table 2 compares some of the methods that have been employed for automatic HS detection in the literature.…”

Section: Related Workmentioning

confidence: 99%

T5 for Hate Speech, Augmented Data, and Ensemble

Adewumi,

Sabry,

Abid

et al. 2023

Sci

View full text Add to dashboard Cite

We conduct relatively extensive investigations of automatic hate speech (HS) detection using different State-of-The-Art (SoTA) baselines across 11 subtasks spanning six different datasets. Our motivation is to determine which of the recent SoTA models is best for automatic hate speech detection and what advantage methods, such as data augmentation and ensemble, may have on the best model, if any. We carry out six cross-task investigations. We achieve new SoTA results on two subtasks—macro F1 scores of 91.73% and 53.21% for subtasks A and B of the HASOC 2020 dataset, surpassing previous SoTA scores of 51.52% and 26.52%, respectively. We achieve near-SoTA results on two others—macro F1 scores of 81.66% for subtask A of the OLID 2019 and 82.54% for subtask A of the HASOC 2021, in comparison to SoTA results of 82.9% and 83.05%, respectively. We perform error analysis and use two eXplainable Artificial Intelligence (XAI) algorithms (Integrated Gradient (IG) and SHapley Additive exPlanations (SHAP)) to reveal how two of the models (Bi-Directional Long Short-Term Memory Network (Bi-LSTM) and Text-to-Text-Transfer Transformer (T5)) make the predictions they do by using examples. Other contributions of this work are: (1) the introduction of a simple, novel mechanism for correcting Out-of-Class (OoC) predictions in T5, (2) a detailed description of the data augmentation methods, and (3) the revelation of the poor data annotations in the HASOC 2021 dataset by using several examples and XAI (buttressing the need for better quality control). We publicly release our model checkpoints and codes to foster transparency.

show abstract

“…Despite such systems demonstrating high accuracy in detecting harmful content, they are not near-perfect and often lack consideration for children's viewpoints in their design. Notably, studies focusing on explainable filtering of online bullying text content have made strides in computational research [15], [16], but fall short in incorporating children's perspectives to enhance the transparency of the AI filtering system's decision-making process.…”

Section: Ai Algorithmic Opaquenessmentioning

confidence: 99%

Designing Ethical Artificial Intelligence (Ai) Systems With Meaningful Youth Participation: Implications and Considerations

Verma,

Milosevic,

Davis

et al. 2023

SPIR

View full text Add to dashboard Cite

While artificial intelligence (AI) enabled systems have shown impressive accuracy in detecting harmful content online, they are still not perfect and do not take into account the perspective of children in their design. The development of AI systems heavily relies on large datasets for training, and creating such datasets involves annotating vast amounts of data. Studies that involve children in dataset development also have their challenges, such as the possibility of re-traumatisation. Therefore, ethical considerations must be taken into account, such as obtaining informed consent, conducting design sessions with children and young people, and addressing implicit and explicit biases in AI filtering, profiling, and surveillance systems. It is crucial to involve children and young people in the design of AI systems that filter content to ensure ethical considerations are met. In this article we discuss the ethical concerns in AI development with children and young people, and also possible techniques that help mitigate such concerns.

show abstract

Does BERT Pay Attention to Cyberbullying?

Cited by 15 publications

References 30 publications

Darkness can not drive out darkness: Investigating Bias in Hate SpeechDetection Models

Darkness can not drive out darkness: Investigating Bias in Hate SpeechDetection Models

T5 for Hate Speech, Augmented Data, and Ensemble

Designing Ethical Artificial Intelligence (Ai) Systems With Meaningful Youth Participation: Implications and Considerations

Contact Info

Product

Resources

About