Visual Named Entity Linking: A New Dataset and A Baseline

Sun, Wenping; Fan, Yixing; Guo, Jiafeng; Zhang, Ruqing; Cheng, Xueqi

doi:10.48550/arxiv.2211.04872

Cited by 1 publication

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this chapter, we have selected a variety of representative approaches for evaluation. These include CLIP4Clip (Luo et al, 2022) in the domain of video retrieval, a purely textual entity linking approach BLINK (Logeswaran et al, 2019), the multimodal entity linking method V2VTEL (Sun et al, 2022), and other multimodal retrieval methods such as AltCLIP and Chinese-CLIP. The experimental metrics primarily utilized are Recall and Mean Reciprocal Rank (MRR) at K. The experimental outcomes are as exhibited in the Table 2.…”

Section: Static Experiments Resultsmentioning

confidence: 99%

“…Existing research primarily focuses on static image-text pairs. Researchers (Adjali et al, 2020a,b;Zhou et al, 2021;Wang et al, 2022b,a;Gan et al, 2021;Sun et al, 2022;Chengmei et al, 2023;Xing et al, 2023;Yao et al, 2023;Zhang et al, 2021) constructed multiple datasets for different scenarios or proposed various multimodal representation methods, integrating features from different modalities to facilitate entity mention and entity matching.…”

Section: Multi-modal Entity Linkingmentioning

confidence: 99%

“…Related research on this task is still relatively limited. There have been studies (Adjali et al, 2020a,b;Zhou et al, 2021;Wang et al, 2022b,a;Gan et al, 2021;Sun et al, 2022;Luo et al, 2023;Xing et al, 2023) focused on the research of Multimodal Entity Linking (MEL), which aims to link mentions of multiple modalities (primarily text and images) to a knowledge base. These works primarily focus on static visual-textual pairs, with limited consideration for mentions in video data.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Nucleic Acids Analysis

et al. 2020

View full text Add to dashboard Cite

In recent years, multi-modal entity linking (MEL) has garnered increasing attention in the research community due to its significance in numerous multi-modal applications. Video, as a popular means of information transmission, has become prevalent in people's daily lives. However, most existing MEL methods primarily focus on linking textual and visual mentions or offline videos's mentions to entities in multi-modal knowledge bases, with limited efforts devoted to linking mentions within online video content. In this paper, we propose a task called Online Video Entity Linking (OVEL), aiming to establish connections between mentions in online videos and a knowledge base with high accuracy and timeliness. To facilitate the research works of OVEL, we specifically concentrate on live delivery scenarios and construct a live delivery entity linking dataset called LIVE. Besides, we propose an evaluation metric that considers timelessness, robustness, and accuracy. Furthermore, to effectively handle OVEL task, we leverage a memory block managed by a Large Language Model and retrieve entity candidates from the knowledge base to augment LLM performance on memory management. The experimental results prove the effectiveness and efficiency of our method.

show abstract

Section: Static Experiments Resultsmentioning

confidence: 99%