CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition

Abstract

The attention-based encoder-decoder framework is becoming popular in scene text recognition, largely due to its superiority in integrating recognition clues from both visual and semantic domains. However, recent studies show the two clues might be misaligned in the difficult text (e.g., with rare text shapes) and introduce constraints such as character position to alleviate the problem. Despite certain success, a content-free positional embedding hardly associates with meaningful local image regions stably. In this paper, we propose a novel module called Multi-Domain Character Distance Perception (MDCDP) to establish a visual and semantic related position encoding. MDCDP uses positional embedding to query both visual and semantic features following the attention mechanism. It naturally encodes the positional clue, which describes both visual and semantic distances among characters. We develop a novel architecture named CDistNet that stacks MDCDP several times to guide precise distance modeling. Thus, the visual-semantic alignment is well built even various difficulties presented. We apply CDistNet to two augmented datasets and six public benchmarks. The experiments demonstrate that CDis-tNet achieves state-of-the-art recognition accuracy. While the visualization also shows that CDistNet achieves proper attention localization in both visual and semantic domains. The code will be released in https://github.com/ simplify23/CDistNet.

Collaborative Encoding Method for Scene Text Recognition in Low Linguistic Resources: The Uyghur Language Case Study

“…Additionally, classic methods for feature extraction using CNN, such as CRNN [38] and TRBA [1], were also included. Additionally, we incorporated the CDistNet method, which combines CNN and Transformer for joint feature extraction [52]. To serve as the baseline for this article, we introduced the PARSeq method [53], involving the joint learning of internal language models.…”

“…Existing internal language joint learning methods typically employ the Transformer architecture and integrate corresponding language branches. SRN [51] employs a semantic reasoning network to assist in text recognition, while CDistNet [52] introduces positional query vectors to align visual and semantic features. PARSeq [53] achieves decoding for arbitrary methods by using Permutation Language Modeling, creating connections between arbitrary characters.…”

Text Recognition Model Based on Multi-Scale Fusion CRNN

“…The model proposed by He et al [ 53 ] constructs a subgraph for each instance and trains it using graph convolutional network and cross-entropy loss function, which achieves good results in text recognition. Zheng et al proposed a new multi-domain character distance perception (MDCDP) module [ 54 ] to establish visually and semantically relevant position encoding, so as to improve the recognition position of the model. Cui et al proposed a representation and correlation enhanced encoder–decoder framework (RCEED) [ 55 ] to address these shortcomings and break through the performance bottleneck.…”

SUST and RUST: Two Datasets for Uyghur Scene Text Recognition

“…The NRTR algorithm [27] advocates full application of the Transformer architecture for image encoding and decoding, incorporating a simple convolution layer for feature extraction. CDistNet [39] developed a specialized module for location modeling to aid character decoding using accurate image features. Nonetheless, these techniques employ a step-by-step decoding approach, recognizing characters individually, leading to reduced speed.…”