Visual Matching is Enough for Scene Text Retrieval

Wen, Lilong; Wang, Yingrong; Zhang, Dongxiang; Chen, Gang

doi:10.1145/3539597.3570428

Cited by 8 publications

(33 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The recent approach [18] makes full use of visual features for matching and achieves state-of-the-art performance. The method converts the text on the query side character by character to an image, which is downsampled for visual embedding.…”

Section: Related Workmentioning

confidence: 99%

“…This branch aims to obtain the visual features of the input query. Firstly, we convert the entire query text into an image, with the conversion method derived from [18]. In the experiments, we use Arial-Unicode as the default font, set the font size to 32×32, the font color to black, and the background to white.…”

Section: Query Image Embedding Branchmentioning

confidence: 99%

“…These augmentation techniques are derived from research in text recognition [26]. Additionally, similar to [18], we use multiple fonts to enrich the styles of query images. However, during the evaluation process, we generate standard query images using the default font.…”

Section: Query Image Embedding Branchmentioning

confidence: 99%

“…However, these methods have limitations as they require pre-cropped text images as input. End-to-end methods [17,18] address this issue by simultaneously locating text regions in the image and embedding the feature representation of these regions, optimizing a common embedding space. The Levenshtein distance [19] is used as the target value to optimize the cosine similarity between the query text and text regions, achieving cross-modal matching.…”

Section: Introductionmentioning

confidence: 99%

“…Finally, the visual features of the query image and text instances are outputted through a Visual Embedding Module (VEM). Additionally, unlike [17,18], we use an improved edit distance [19] and loss function to optimize our model. The main contributions of this paper are summarized as follows:…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Visual Information Facilitation Scene Text Retrieval

Luo,

Ibrayim,

Hamdulla

et al. 2024

Preprint

View full text Add to dashboard Cite

Given a query text and a database of images to be queried, a scene text retrieval system can output the addresses of images containing this query word, as well as the location of the query word in the images. The current state-of-the-art method matches the visual features of query embedding and text instance embedding. This approach effectively mitigates the heterogeneity differences between the two modalities, but it disrupts the visual structure of the query images. In this paper, we optimize this method. First, we directly convert the entire query text into an image, thereby maximally preserving the visual information of the original text image. Then, we devised a Coarse-Grained Feature Rectification (CGFR) module, which facilitates visual alignment. Finally, we propose Adaptive Edit Distance (AED) and improve the main loss function. With the promotion of the above scheme, our method has reached the state-of-the-art level on multiple benchmark datasets. Particularly, this method is suitable for multilingual retrieval tasks, with a 16.98% improvement in mAP score compared to the current state-of-the-art method.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Query Image Embedding Branchmentioning

confidence: 99%