Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrieval

Li, Pandeng; Xie, Hongtao; Ge, Jiannan; Zhang, Lei; Min, Shaobo; Zhang, Yongdong

doi:10.1007/978-3-031-19781-9_11

Cited by 11 publications

(1 citation statement)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…General knowledge distillation methods [16,21] use the output of the teacher model to generate soft guidance and force the student model to predict similar results. With the development of largescale pre-trained models, many works utilize pre-trained models to distill their model.…”

Section: Knowledge Distillationmentioning

confidence: 99%

Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognition

Wang,

Xie,

Wang

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

In this paper, we explore the potential of the Contrastive Language-Image Pretraining (CLIP) model in scene text recognition (STR), and establish a novel Symmetrical Linguistic Feature Distillation framework (named CLIP-OCR) to leverage both visual and linguistic knowledge in CLIP. Different from previous CLIP-based methods mainly considering feature generalization on visual encoding, we propose a symmetrical distillation strategy (SDS) that further captures the linguistic knowledge in the CLIP text encoder. By cascading the CLIP image encoder with the reversed CLIP text encoder, a symmetrical structure is built with an image-to-text feature flow that covers not only visual but also linguistic information for distillation. Benefiting from the natural alignment in CLIP, such guidance flow provides a progressive optimization objective from vision to language, which can supervise the STR feature forwarding process layer-by-layer. Besides, a new Linguistic Consistency Loss (LCL) is proposed to enhance the linguistic capability by considering second-order statistics during the optimization. Overall, CLIP-OCR is the first to design a smooth transition between image and text for the STR task. Extensive experiments demonstrate the effectiveness of CLIP-OCR with 93.8% average accuracy on six popular STR benchmarks. Code will be available at https://github.com/wzx99/CLIPOCR. CCS CONCEPTS• Applied computing → Optical character recognition.

show abstract

Section: Knowledge Distillationmentioning

confidence: 99%