Dynamically Disentangling Social Bias from Task-Oriented Representations with Adversarial Attack

Wang, Liwen; Yan, Yuanmeng; He, Keqing; Wu, Yanan; Wang, Xu

doi:10.18653/v1/2021.naacl-main.293

Cited by 18 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As there is still a huge gap in model performance between unseen and seen slots, in future work, we will focus on improving the performance on unseen slots while maintaining the performance on seen slots. Representation disentanglement (Wang et al, 2021) can be used to disentangle domain-specific and domain-shared knowledge in the source domain, then we can preserve domainshared knowledge and focus on establishing the relation of domain-specific knowledge between the source and the target domain.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Wang¹,

Li²,

Liu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Zero-shot cross-domain slot filling alleviates the data dependence in the case of data scarcity in the target domain, which has aroused extensive research. However, as most of the existing methods do not achieve effective knowledge transfer to the target domain, they just fit the distribution of the seen slot and show poor performance on unseen slot in the target domain. To solve this, we propose a novel approach based on prototypical contrastive learning with a dynamic label confusion strategy for zero-shot slot filling. The prototypical contrastive learning aims to reconstruct the semantic constraints of labels, and we introduce the label confusion strategy to establish the label dependence between the source domains and the target domain on-the-fly. Experimental results show that our model achieves significant improvement on the unseen slots, while also set new state-of-the-arts on slot filling task. 1

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Wang¹,

Li²,

Liu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several works further investigate the use of adversarial training for removing demographic information from neural/PLM-based text classifiers (Elazar and Goldberg, 2018;Barrett et al, 2019;Wang et al, 2021). Notably, Han et al (2021b) show the benefit of having an ensemble of orthogonal adversaries.…”

Section: Fairness and Bias Mitigation In Nlpmentioning

confidence: 99%

Parameter-efficient Modularised Bias Mitigation via AdapterFusion

Kumar,

Lesota,

Zerveas

et al. 2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Large pre-trained language models contain societal biases and carry along these biases to downstream tasks. Current in-processing bias mitigation approaches (like adversarial training) impose debiasing by updating a model's parameters, effectively transferring the model to a new, irreversible debiased state. In this work, we propose a novel approach to develop stand-alone debiasing functionalities separate from the model, which can be integrated into the model on-demand, while keeping the core model untouched. Drawing from the concept of AdapterFusion in multi-task learning, we introduce DAM (Debiasing with Adapter Modules) -a debiasing approach to first encapsulate arbitrary bias mitigation functionalities into separate adapters, and then add them to the model on-demand in order to deliver fairness qualities. We conduct a large set of experiments on three classification tasks with gender, race, and age as protected attributes. Our results show that DAM improves or maintains the effectiveness of bias mitigation, avoids catastrophic forgetting in a multi-attribute scenario, and maintains on-par task performance, while granting parameter-efficiency and easy switching between the original and debiased models.

show abstract

“…In order to enhance the robustness of the model, we introduce adversarial training [15] [16] to reduce the influence of domain noise. First, x i is encoded to obtain the representation h i , and the negative log-likelihood (NLL) loss is calculated as follows:…”

Section: Adversarial Learningmentioning

confidence: 99%

A Robust Contrastive Alignment Method For Multi-Domain Text Classification

Li¹,

Liang²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Multi-domain text classification can automatically classify texts in various scenarios. Due to the diversity of human languages, texts with the same label in different domains may differ greatly, which brings challenges to the multidomain text classification. Current advanced methods use the private-shared paradigm, capturing domain-shared features by a shared encoder, and training a private encoder for each domain to extract domain-specific features. However, in realistic scenarios, these methods suffer from inefficiency as new domains are constantly emerging. In this paper, we propose a robust contrastive alignment method to align text classification features of various domains in the same feature space by supervised contrastive learning. By this means, we only need two universal feature extractors to achieve multidomain text classification. Extensive experimental results show that our method performs on par with or sometimes better than the state-of-the-art method, which uses the complex multi-classifier in a private-shared framework.

show abstract

Dynamically Disentangling Social Bias from Task-Oriented Representations with Adversarial Attack

Cited by 18 publications

References 29 publications

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Parameter-efficient Modularised Bias Mitigation via AdapterFusion

A Robust Contrastive Alignment Method For Multi-Domain Text Classification

Contact Info

Product

Resources

About