From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil Transfer

Pavlopoulos, John; Laugier, Léo; Xenos, Alexandros; Sorensen, Jeffrey; Androutsopoulos, Ion

doi:10.18653/v1/2022.acl-long.259

Cited by 10 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extensive prior works attempted to define and predict inappropriate behaviors online, yet framed in various ways, such as explicit attacks (Wulczyn, Thain, and Dixon 2016), online abuse (Mishra, Yannakoudakis, and Shutova 2019) toxicity (Pavlopoulos, Malakasiotis, and Androutsopoulos 2017;Zampieri et al 2019;Pavlopoulos et al 2022;Brassard-Gourdeau and Khoury 2019), and hate speech (Davidson et al 2017;Schmidt and Wiegand 2017;Mosca, Wich, and Groh 2021). For example, Davidson et al (2017) trained a multi-class model to differentiate between everyday offensive language and serious hate speech.…”

Section: Quantifying Incivility and Similar Conceptsmentioning

confidence: 99%

A Crisis of Civility? Modeling Incivility and Its Effects in Political Discourse Online

Gao,

Qin,

Murali

et al. 2024

ICWSM

View full text Add to dashboard Cite

Growing concerns have been raised about the detrimental effects of uncivil comments on the web towards democracy. However, there is still a lack of understanding about online incivility's nuanced and complicated nature and its impact on conversation development and user behaviors. This work aims to fill that research gap by modeling incivility and its relationship to political discussions. We develop a comprehensive and fine-grained taxonomy that characterizes incivility with vulgarity, name-calling (inter-personal and third-party attacks), aspersion, and stereotypes, and then apply the framework to quantify the level of each incivility category in over 40 million comments from Reddit. Using large-scale quantitative analysis, we investigate the types of interactions and contexts in which incivility is more likely to occur, model how incivility shapes subsequent conversations, and examine user engagement patterns and behavioral changes after exposure to incivility. Our findings show that conversations that start out uncivil tend to become more uncivil in responses, and exposure to different incivility categories has differing effects on community members' engagement. We conclude with the implications of our research in assisting the design and moderation of online political communities.

show abstract

Section: Quantifying Incivility and Similar Conceptsmentioning

confidence: 99%

A Crisis of Civility? Modeling Incivility and Its Effects in Political Discourse Online

Gao,

Qin,

Murali

et al. 2024

ICWSM

View full text Add to dashboard Cite

show abstract

“…Most of the research conducted in the area of toxic span classification was on English language corpora. This research began with the aforementioned Semeval2021 task or with its follow-up work on detoxifying posts [27] using the ToxicSpans dataset. Many of these studies focused on the explainability of existing classification methods [28,29].…”

Section: Offensive and Toxic Spans' Datasetsmentioning

confidence: 99%

Offensive Text Span Detection in Romanian Comments Using Large Language Models

Paraschiv,

Ion,

Dascalu

2023

Information

View full text Add to dashboard Cite

The advent of online platforms and services has revolutionized communication, enabling users to share opinions and ideas seamlessly. However, this convenience has also brought about a surge in offensive and harmful language across various communication mediums. In response, social platforms have turned to automated methods to identify offensive content. A critical research question emerges when investigating the role of specific text spans within comments in conveying offensive characteristics. This paper conducted a comprehensive investigation into detecting offensive text spans in Romanian language comments using Transformer encoders and Large Language Models (LLMs). We introduced an extensive dataset of 4800 Romanian comments annotated with offensive text spans. Moreover, we explored the impact of varying model sizes, architectures, and training data volumes on the performance of offensive text span detection, providing valuable insights for determining the optimal configuration. The results argue for the effectiveness of BERT pre-trained models for this span-detection task, showcasing their superior performance. We further investigated the impact of different sample-retrieval strategies for few-shot learning using LLMs based on vector text representations. The analysis highlights important insights and trade-offs in leveraging LLMs for offensive-language-detection tasks.

show abstract

“…In the explanations as input experiments (see section 5), the complete post (tokenized) is taken as their rationale for the normal posts. ToxicSpans dataset: The ToxicSpans (Pavlopoulos et al, 2022) dataset is a subset (containing 11,006 samples labelled toxic) of the Civil Comments dataset (1.2M Posts). The dataset also contains the toxic spans, i.e., the region of the texts found toxic.…”

Section: Datasets and Metricsmentioning

confidence: 99%

“…We, for the first time, introduce several prompt variations and input instructions to probe two of the LLMs (GPT 3.5 and text-davinci) across three datasets -HateXplain (Mathew et al, 2021), implicit hate (ElSherief et al, 2021) and Toxic-Spans (Pavlopoulos et al, 2022). Note that all these three datasets contain ground truth explanations in the form of either rationales (Mathew et al, 2021;Pavlopoulos et al, 2022) or implied statements (ElSherief et al, 2021) that tells why an annotator took a particular labelling decision. In addition, two of the datasets also contain the information about the target/victim community against whom the hate speech was hurled.…”

Section: Introductionmentioning

confidence: 99%

Probing LLMs for hate speech detection: strengths and vulnerabilities

Roy,

Harshvardhan,

Mukherjee

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Recently efforts have been made by social media platforms as well as researchers to detect hateful or toxic language using large language models. However, none of these works aim to use explanation, additional context and victim community information in the detection process. We utilise different prompt variation, input information and evaluate large language models in zero shot setting (without adding any in-context examples). We select three large language models (GPT-3.5, text-davinci and Flan-T5) and three datasets -HateXplain, implicit hate and ToxicSpans. We find that on average including the target information in the pipeline improves the model performance substantially (∼ 20−30%) over the baseline across the datasets. There is also a considerable effect of adding the rationales/explanations into the pipeline (∼ 10 − 20%) over the baseline across the datasets. In addition, we further provide a typology of the error cases where these large language models fail to (i) classify and (ii) explain the reason for the decisions they take. Such vulnerable points automatically constitute 'jailbreak' prompts for these models and industry scale safeguard techniques need to be developed to make the models robust against such prompts.

show abstract

From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil Transfer

Cited by 10 publications

References 23 publications

A Crisis of Civility? Modeling Incivility and Its Effects in Political Discourse Online

A Crisis of Civility? Modeling Incivility and Its Effects in Political Discourse Online

Offensive Text Span Detection in Romanian Comments Using Large Language Models

Probing LLMs for hate speech detection: strengths and vulnerabilities

Contact Info

Product

Resources

About