Avoiding Unintended Bias in Toxicity Classification with Neural Networks

Morzhov, Sergey V.

doi:10.23919/fruct48808.2020.9087368

Cited by 13 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning technologies have been leveraged by the authors in [40] to tackle the problem of toxic comments detection. More in details, the authors introduced two state-ofthe-art neural network architectures and demonstrate how to employ a contextual language representation model.…”

Section: Related Workmentioning

confidence: 99%

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

2021

View full text Add to dashboard Cite

Today, increasing numbers of people are interacting online and a lot of textual comments are being produced due to the explosion of online communication. However, a paramount inconvenience within online environments is that comments that are shared within digital platforms can hide hazards, such as fake news, insults, harassment, and, more in general, comments that may hurt someone’s feelings. In this scenario, the detection of this kind of toxicity has an important role to moderate online communication. Deep learning technologies have recently delivered impressive performance within Natural Language Processing applications encompassing Sentiment Analysis and emotion detection across numerous datasets. Such models do not need any pre-defined hand-picked features, but they learn sophisticated features from the input datasets by themselves. In such a domain, word embeddings have been widely used as a way of representing words in Sentiment Analysis tasks, proving to be very effective. Therefore, in this paper, we investigated the use of deep learning and word embeddings to detect six different types of toxicity within online comments. In doing so, the most suitable deep learning layers and state-of-the-art word embeddings for identifying toxicity are evaluated. The results suggest that Long-Short Term Memory layers in combination with mimicked word embeddings are a good choice for this task.

show abstract

Section: Related Workmentioning

confidence: 99%

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

2021

View full text Add to dashboard Cite

show abstract

“…It's worth noting that the Perspective API has been updated, and as of February 2020, its performance in terms of toxicity classification is significantly improved, approaching the results of other advanced models such as the Bi-GRU with attention mechanism model. This update enhances the effectiveness of Perspective in identifying and addressing toxic content in real-time social media interactions [48].…”

Section: G) Toxicity Analysismentioning

confidence: 88%

Topic prediction for tobacco control based on COP9 tweets using machine learning techniques

Elmitwalli,

Mehegan,

Wellock

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

The prediction of tweets associated with specific topics offers the potential to automatically focus on and understand online discussions surrounding these issues. This paper introduces a comprehensive approach that centers on the topic of "harm reduction" within the broader context of tobacco control. The study leveraged tweets from the period surrounding the ninth Conference of the Parties to review the Framework Convention on Tobacco Control (COP9) as a case study to pilot this approach. By using Latent Dirichlet Allocation (LDA)-based topic modeling, the study successfully categorized tweets related to harm reduction. Subsequently, various machine learning techniques were employed to predict these topics, achieving a prediction accuracy of 91.87% using the Random Forest algorithm. Additionally, the study explored correlations between retweets and sentiment scores. It also conducted a toxicity analysis to understand the extent to which online conversations lacked neutrality. Understanding the topics, sentiment, and toxicity of Twitter data is crucial for identifying public opinion and its formation. By specifically focusing on the topic of “harm reduction” in tweets related to COP9, the findings offer valuable insights into online discussions surrounding tobacco control. This understanding can aid policymakers in effectively informing the public and garnering public support, ultimately contributing to the successful implementation of tobacco control policies.

show abstract

“…Comparative analyses predominantly occur between newly proposed methodologies and more conventional approaches or with established state-of-the-art models, even if the latter were not necessarily trained on the pertinent data sources [34,35,71,114,138,207,212]. One of the discernible outcomes delineated in existing literature suggests that transformer-based models, when juxtaposed with conventional machine learning methodologies, demonstrate superior performance, albeit often by a marginal degree [42,57,60,76,82,143,208]. Moreover, it has been noted that models trained within the specific domain or those pre-trained on domain-contextualized datasets tend to exhibit enhanced performance compared to their counterparts lacking such contextual embeddings [55,64,92].…”

Section: Toxicity At Sentence Level Classificationmentioning

confidence: 99%

“…Furthermore, researchers have utilized masked language modelling techniques to assess a model's ability to detect toxicity given a prompt and a template question [56], with findings indicating the superiority of T5 over GPT-2 in this regard. Nevertheless, it has been recognized that a widely adopted strategy for achieving improved performance entails the utilization of ensemble methods [53,57,77,129].…”

Section: Toxicity At Sentence Level Classificationmentioning

confidence: 99%

A Systematic Review of Toxicity in Large Language Models: Definitions, Datasets, Detectors, Detoxification Methods and Challenges

Villate-Castillo,

Ser,

Urquijo

2024

Preprint

View full text Add to dashboard Cite

The emergence of the transformer architecture has ushered in a new era of possibilities, showcasing remarkable capabilities in generative tasks exemplified by models like GPT4o, Claude 3, and Llama 3. However, these advancements come with a caveat: predominantly trained on data gleaned from social media platforms, these systems inadvertently perpetuate societal biases and toxicity. Recognizing the paramount importance of AI Safety and Alignment, our study embarks on a thorough exploration through a comprehensive literature review focused on toxic language. Delving into various definitions, detection methodologies, and mitigation strategies, we aim to shed light on the complexities of this issue. While our focus primarily centres on transformer-based architectures, we also acknowledge and incorporate existing research within the realm of deep learning. Through our investigation, we uncover a multitude of challenges inherent in toxicity mitigation and detection models. These challenges range from inherent biases and generalization issues to the necessity for standardized definitions of toxic language and the quality assurance of dataset annotations. Furthermore, we emphasize the significance of transparent annotation processes, resolution of annotation disagreements, and the enhancement of Large Language Models (LLMs) robustness. Additionally, we advocate for the creation of standardized benchmarks to gauge the effectiveness of toxicity mitigation and detection methods. Addressing these challenges is not just imperative, but pivotal in advancing the development of safer and more ethically aligned AI systems.

show abstract

Avoiding Unintended Bias in Toxicity Classification with Neural Networks

Cited by 13 publications

References 9 publications

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

Topic prediction for tobacco control based on COP9 tweets using machine learning techniques

A Systematic Review of Toxicity in Large Language Models: Definitions, Datasets, Detectors, Detoxification Methods and Challenges

Contact Info

Product

Resources

About