DAMO-NLP at NLPCC-2022 Task 2: Knowledge Enhanced Robust NER for Speech Entity Linking

Abstract

Speech Entity Linking aims to recognize and disambiguate named entities in spoken languages. Conventional methods suffer gravely from the unfettered speech styles and the noisy transcripts generated by ASR systems. In this paper, we propose a novel approach called Knowledge Enhanced Named Entity Recognition (KENER), which focuses on improving robustness through painlessly incorporating proper knowledge in the entity recognition stage and thus improving the overall performance of entity linking. KENER first retrieves candidate entities for a sentence without mentions, and then utilizes the entity descriptions as extra information to help recognize mentions. The candidate entities retrieved by a dense retrieval module are especially useful when the input is short or noisy. Moreover, we investigate various data sampling strategies and design effective loss functions, in order to improve the quality of retrieved entities in both recognition and disambiguation stages. Lastly, a linking with filtering module is applied as the final safeguard, making it possible to filter out wrongly-recognized mentions. Our system achieves 1st place in Track 1 and 2nd place in Track 2 of NLPCC-2022 Shared Task 2.

Attention-enabled Multi-layer Subword Joint Learning for Chinese Word Embedding

“…Amidst the rapid ascent of generative big model technology, a plethora of advanced pre-training word vector methodologies have emerged, encompassing, but not confined to: the MSE model [23], adept at entity linking tasks, aimed at disambiguating named entities within spoken language; the Generalized Word Vector Model CoROM [24], and the GTE model [25] developed by the Tongyi Lab; the BCE [26] model by Youdao, renowned for its robust bilingual and cross-linguistic semantic characterization prowess; and the M3E model [27] by the Moka team. These expansive word vectors, trained on colossal corpora, not only offer heightened accuracy and semantic richness at the word level but also exhibit robust generalization capabilities, demonstrating exceptional performance across a myriad of natural language processing tasks.…”

Generative Multi-Modal Knowledge Retrieval with Large Language Models

“…Knowledge Retrieval (KR) is crucial in supporting knowledge-intensive multi-modal applications, such as visual question answering (VQA) (Ma et al 2023), multimodal entity linking (Huang et al 2022) and multi-modal dialogue (Ma et al 2022). In these applications, the information available within the multi-modal contexts may be insufficient, necessitating the acquisition of external knowledge.…”