Cross-Lingual Named Entity Recognition for Heterogenous Languages

Fu, Yingwen; Lin, Nankai; Chen, Boyu; Yang, Ziyu; Jiang, Shengyi

doi:10.1109/taslp.2022.3212698

Cited by 2 publications

(2 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-Lingual NER Various strategies have been investigated to tackle the zero-resource challenge for cross-lingual NER, such as translation-based (Xie et al 2018;Liang et al 2021), direct transfer-based (Wu and Dredze 2019;Wu et al 2020c), and knowledge distillation-based approaches (Wu et al 2020a;Liang et al 2021;Chen et al 2021;Zeng et al 2022;Fu et al 2022). The direct transfer-based methods yield inferior results, since the target data has not been effectively utilized.…”

Section: Related Workmentioning

confidence: 99%

“…Wu et al (2020a) pioneers the use of knowledge distillation architectures for crosslingual NER tasks and achieves surprising performance. Since then, researchers in this field have focused on knowledge distillation-based approaches and developed various improvements (Zeng et al 2022;Fu et al 2022). Following the previous works above, this paper employs the knowledge distillation architecture as the backbone for cross-lingual NER.…”

Section: Knowledge Distillationmentioning

confidence: 99%

See 1 more Smart Citation

Discrepancy and Uncertainty Aware Denoising Knowledge Distillation for Zero-Shot Cross-Lingual Named Entity Recognition

Ge,

Hu,

et al. 2024

AAAI

View full text Add to dashboard Cite

The knowledge distillation-based approaches have recently yielded state-of-the-art (SOTA) results for cross-lingual NER tasks in zero-shot scenarios. These approaches typically employ a teacher network trained with the labelled source (rich-resource) language to infer pseudo-soft labels for the unlabelled target (zero-shot) language, and force a student network to approximate these pseudo labels to achieve knowledge transfer. However, previous works have rarely discussed the issue of pseudo-label noise caused by the source-target language gap, which can mislead the training of the student network and result in negative knowledge transfer. This paper proposes an discrepancy and uncertainty aware Denoising Knowledge Distillation model (DenKD) to tackle this issue. Specifically, DenKD uses a discrepancy-aware denoising representation learning method to optimize the class representations of the target language produced by the teacher network, thus enhancing the quality of pseudo labels and reducing noisy predictions. Further, DenKD employs an uncertainty-aware denoising method to quantify the pseudo-label noise and adjust the focus of the student network on different samples during knowledge distillation, thereby mitigating the noise's adverse effects. We conduct extensive experiments on 28 languages including 4 languages not covered by the pre-trained models, and the results demonstrate the effectiveness of our DenKD.

show abstract