Toxicity Detection in Online Comments with Limited Data: A Comparative Analysis

Lübbering, Max; Pielka, Maren; Das, Kajaree; Gebauer, Michael; Ramamurthy, Rajkumar; Bauckhage, Christian; Sifa, Rafet

doi:10.14428/esann/2021.es2021-48

Cited by 3 publications

(1 citation statement)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the related OCC setting with its focus on outlier detection, DNN-based approaches have been researched from three angles: (1) combining kernel methods [65] with DNN methods [14,60,71], (2) generative models (e.g., generative adversarial networks [22] or variational autoencoders [35])based outlier detectors [50,64,72], and (3) based on (semi-) supervised autoencoders [8,10,26,44,45,47]. Here, the key idea is to learn a representation of the inlier distribution and subsequently to estimate the outlierness of a sample via its reconstruction error.…”

Section: Related Workmentioning

confidence: 99%

Bounding open space risk with decoupling autoencoders in open set recognition

Lübbering

Gebauer

Ramamurthy

et al. 2022

Int J Data Sci Anal

Self Cite

View full text Add to dashboard Cite

One-vs-Rest (OVR) classification aims to distinguish a single class of interest (COI) from other classes. The concept of novelty detection and robustness to dataset shift becomes crucial in OVR when the scope of the rest class is extended from the classes observed during training to unseen and possibly unrelated classes, a setting referred to as open set recognition (OSR). In this work, we propose a novel architecture, namely decoupling autoencoder (DAE), which provides a proven upper bound on the open space risk and minimizes open space risk via a dedicated training routine. Our method is benchmarked within three different scenarios, each isolating different aspects of OSR, namely plain classification, outlier detection, and dataset shift. The results conclusively show that DAE achieves robust performance across all three tasks. This level of cross-task robustness is not observed for any of the seven potent baselines from the OSR, OVR, outlier detection, and ensembling domain which, apart from ATA (Lübbering et al., From imbalanced classification to supervised outlier detection problems: adversarially trained auto encoders. In: Artificial neural networks and machine learning—ICANN 2020, 2020), tend to fail on either one of the tasks. Similar to DAE, ATA is based on autoencoders and facilitates the reconstruction error to predict the inlierness of a sample. However unlike DAE, it does not provide any uncertainty scores and therefore lacks rudimentary means of interpretation. Our adversarial robustness and local stability results further support DAE’s superiority in the OSR setting, emphasizing its applicability in safety-critical systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Bounding open space risk with decoupling autoencoders in open set recognition

Lübbering

Gebauer

Ramamurthy

et al. 2022

Int J Data Sci Anal

Self Cite

View full text Add to dashboard Cite

show abstract

Cyber Bullying and Toxicity Detection Using Machine Learning

Jadhav,

Agarwal,

Shevate

et al. 2023

2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN)

View full text Add to dashboard Cite

A Systematic Review of Toxicity in Large Language Models: Definitions, Datasets, Detectors, Detoxification Methods and Challenges

Villate-Castillo,

Ser,

Urquijo

2024

Preprint

View full text Add to dashboard Cite

The emergence of the transformer architecture has ushered in a new era of possibilities, showcasing remarkable capabilities in generative tasks exemplified by models like GPT4o, Claude 3, and Llama 3. However, these advancements come with a caveat: predominantly trained on data gleaned from social media platforms, these systems inadvertently perpetuate societal biases and toxicity. Recognizing the paramount importance of AI Safety and Alignment, our study embarks on a thorough exploration through a comprehensive literature review focused on toxic language. Delving into various definitions, detection methodologies, and mitigation strategies, we aim to shed light on the complexities of this issue. While our focus primarily centres on transformer-based architectures, we also acknowledge and incorporate existing research within the realm of deep learning. Through our investigation, we uncover a multitude of challenges inherent in toxicity mitigation and detection models. These challenges range from inherent biases and generalization issues to the necessity for standardized definitions of toxic language and the quality assurance of dataset annotations. Furthermore, we emphasize the significance of transparent annotation processes, resolution of annotation disagreements, and the enhancement of Large Language Models (LLMs) robustness. Additionally, we advocate for the creation of standardized benchmarks to gauge the effectiveness of toxicity mitigation and detection methods. Addressing these challenges is not just imperative, but pivotal in advancing the development of safer and more ethically aligned AI systems.

show abstract

Toxicity Detection in Online Comments with Limited Data: A Comparative Analysis

Cited by 3 publications

References 6 publications

Bounding open space risk with decoupling autoencoders in open set recognition

Bounding open space risk with decoupling autoencoders in open set recognition

Cyber Bullying and Toxicity Detection Using Machine Learning

A Systematic Review of Toxicity in Large Language Models: Definitions, Datasets, Detectors, Detoxification Methods and Challenges

Contact Info

Product

Resources

About