Decipherment for Adversarial Offensive Language Detection

Wu, Zhelun; Kambhatla, Nishant; Sarkar, Anoop

doi:10.18653/v1/w18-5119

Cited by 6 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Toxic content detection attempts to identify content that can offend or harm its recipients, including hate speech (Wang 2018), racism (Waseem and Hovy 2016), and offensive language (Wu, Kambhatla, and Sarkar 2018). Given the subjectivity of these categorizations, we do not limit the scope of our work to any specific type and address toxic content in general.…”

Section: Task and Datasetsmentioning

confidence: 99%

Towards Robust Toxic Content Classification

Kurita,

Belova,

Anastasopoulos

2019

Preprint

View full text Add to dashboard Cite

Toxic content detection aims to identify content that can offend or harm its recipients. Automated classifiers of toxic content need to be robust against adversaries who deliberately try to bypass filters. We propose a method of generating realistic model-agnostic attacks using a lexicon of toxic tokens, which attempts to mislead toxicity classifiers by diluting the toxicity signal either by obfuscating toxic tokens through characterlevel perturbations, or by injecting non-toxic distractor tokens. We show that these realistic attacks reduce the detection recall of state-of-the-art neural toxicity detectors, including those using ELMo and BERT, by more than 50% in some cases. We explore two approaches for defending against such attacks. First, we examine the effect of training on synthetically noised data. Second, we propose the Contextual Denoising Autoencoder (CDAE): a method for learning robust representations that uses character-level and contextual information to denoise perturbed tokens. 1 We show that the two approaches are complementary, improving robustness to both character-level perturbations and distractors, recovering a considerable portion of the lost accuracy. Finally, we analyze the robustness characteristics of the most competitive methods and outline practical considerations for improving toxicity detectors.

show abstract

Section: Task and Datasetsmentioning

confidence: 99%

Towards Robust Toxic Content Classification

Kurita,

Belova,

Anastasopoulos

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Computational decipherment based techniques have seen a wide range of applications ranging such as identifying unknown languages and scripts (Hauer and Kondrak, 2016), writing systems (Born et al, 2019(Born et al, , 2021 and lost languages (Snyder et al, 2010;Luo et al, 2019), offensive langauge detection (Wu et al, 2018;Qian et al, 2019), and, more recently, towards improving neural machine translation (Kambhatla et al, 2022). While decipherment has strong connections to cryptography research, we limit the scope of this work to natural language based decipherment.…”

Section: Other Related Workmentioning

confidence: 99%

Decipherment as Regression: Solving Historical Substitution Ciphers by Learning Symbol Recurrence Relations

Kambhatla,

Born,

Sarkar

2023

Findings of the Association for Computational Linguistics: EACL 2023

View full text Add to dashboard Cite

Solving substitution ciphers involves mapping sequences of cipher symbols to fluent text in a target language. This has conventionally been formulated as a search problem, to find the decipherment key using a character-level language model to constrain the search space. This work instead frames decipherment as a sequence prediction task, using a Transformer-based causal language model to learn recurrences between characters in a ciphertext. We introduce a novel technique for transcribing arbitrary substitution ciphers into a common recurrence encoding. By leveraging this technique, we (i) create a large synthetic dataset of homophonic ciphers using random keys, and (ii) train a decipherment model that predicts the plaintext sequence given a recurrence-encoded ciphertext. Our method achieves strong results on synthetic 1:1 and homophonic ciphers, and cracks several real historic homophonic ciphers. Our analysis shows that the model learns recurrence relations between cipher symbols and recovers decipherment keys in its self-attention. 1

show abstract

“…The former relates to purposefully misspelling or otherwise symbolically replacing text (e.g., fvk you, @ssh*l3) to subvert algorithms (Eger et al, 2019;Kurita et al, 2019). Wu et al (2018) show such attacks on toxic content can be effectively deciphered. Word-level attacks are arguably straight-forward for humans, but significantly more challenging to automate-requiring preservation of toxicity; i.e, the semantics of the sentence.…”

Section: Related Workmentioning

confidence: 99%

Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations

Emmery¹,

Kádár²,

Chrupała³

et al. 2022

Preprint

View full text Add to dashboard Cite

A limited amount of studies investigates the role of model-agnostic adversarial behavior in toxic content classification. As toxicity classifiers predominantly rely on lexical cues, (deliberately) creative and evolving language-use can be detrimental to the utility of current corpora and state-of-the-art models when they are deployed for content moderation. The less training data is available, the more vulnerable models might become. This study is, to our knowledge, the first to investigate the effect of adversarial behavior and augmentation for cyberbullying detection. We demonstrate that model-agnostic lexical substitutions significantly hurt classifier performance. Moreover, when these perturbed samples are used for augmentation, we show models become robust against word-level perturbations at a slight trade-off in overall task performance. Augmentations proposed in prior work on toxicity prove to be less effective. Our results underline the need for such evaluations in online harm areas with small corpora. The perturbed data, models, and code are available for reproduction at https://github.com/cmry/augtox.

show abstract

Decipherment for Adversarial Offensive Language Detection

Cited by 6 publications

References 19 publications

Towards Robust Toxic Content Classification

Towards Robust Toxic Content Classification

Decipherment as Regression: Solving Historical Substitution Ciphers by Learning Symbol Recurrence Relations

Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations

Contact Info

Product

Resources

About