An Empirical Study on Explanations in Out-of-Domain Settings

Chrysostomou, George; Αλέτρας, Νικόλαος

doi:10.18653/v1/2022.acl-long.477

Cited by 11 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach shows consistent cross-corpora performance improvements both independently and in combination with pre-defined tokens. Future work includes applying our method on other cross-domain text classification tasks and exploring how explanation faithfulness can be improved in out-of-domain settings (Chrysostomou and Aletras, 2022).…”

Section: Discussionmentioning

confidence: 99%

Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection

Bose¹,

Αλέτρας²,

Illina³

et al. 2022

Findings of the Association for Computational Linguistics: ACL 2022

Self Cite

View full text Add to dashboard Cite

Warning: this paper contains content that may be offensive and distressing.Hate speech classifiers exhibit substantial performance degradation when evaluated on datasets different from the source. This is due to learning spurious correlations between words that are not necessarily relevant to hateful language, and hate speech labels from the training corpus. Previous work has attempted to mitigate this problem by regularizing specific terms from pre-defined static dictionaries. While this has been demonstrated to improve the generalizability of classifiers, the coverage of such methods is limited and the dictionaries require regular manual updates from human experts. In this paper, we propose to automatically identify and reduce spurious correlations using attribution methods with dynamic refinement of the list of terms that need to be regularized during training. Our approach is flexible and improves the cross-corpora performance over previous work independently and in combination with pre-defined dictionaries. 1

show abstract

Section: Discussionmentioning

confidence: 99%

Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection

Bose¹,

Αλέτρας²,

Illina³

et al. 2022

Findings of the Association for Computational Linguistics: ACL 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Yao et al (2021) do not use any list but they require human-provided refinement advice as inputs. Chrysostomou and Aletras (2022a) further show that post-hoc explanation methods might not provide faithful explanations in out-of-domain settings. The contemporaneous work by Attanasio et al (2022) and Bose et al (2022) reduce lexical overfitting automatically with entropy-based attentions and feature attributions, respectively.…”

Section: Introductionmentioning

confidence: 87%

Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection

Bose¹,

Αλέτρας²,

Illina³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Warning: this paper contains content that may be offensive and distressing.State-of-the-art approaches for hate-speech detection usually exhibit poor performance in out-of-domain settings. This occurs, typically, due to classifiers overemphasizing sourcespecific information that negatively impacts its domain invariance. Prior work has attempted to penalize terms related to hatespeech from manually curated lists using feature attribution methods, which quantify the importance assigned to input terms by the classifier when making a prediction. We, instead, propose a domain adaptation approach that automatically extracts and penalizes source-specific terms using a domain classifier, which learns to differentiate between domains, and feature-attribution scores for hatespeech classes, yielding consistent improvements in cross-domain evaluation.

show abstract

“…The ante-hoc explanation approach has also been referred to as a pipeline , select-then-predict (Chrysostomou and Aletras, 2022), and explain-then-predict (Camburu et al, 2018) setup. The annotator model can be trained separately from the ante-hoc explainer model (Yessenalina et al, 2010;Jain et al, 2020) or the models can be trained jointly (Lei et al, 2016;Bastings et al, 2019).…”

Section: Figurementioning

confidence: 99%

Human-annotated rationales and explainable text classification: a survey

Herrewijnen,

Nguyen,

Bex

et al. 2024

Front. Artif. Intell.

View full text Add to dashboard Cite

Asking annotators to explain “why” they labeled an instance yields annotator rationales: natural language explanations that provide reasons for classifications. In this work, we survey the collection and use of annotator rationales. Human-annotated rationales can improve data quality and form a valuable resource for improving machine learning models. Moreover, human-annotated rationales can inspire the construction and evaluation of model-annotated rationales, which can play an important role in explainable artificial intelligence.

show abstract

An Empirical Study on Explanations in Out-of-Domain Settings

Cited by 11 publications

References 0 publications

Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection

Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection

Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection

Human-annotated rationales and explainable text classification: a survey

Contact Info

Product

Resources

About