Dictionary-guided Scene Text Recognition

Nguyen, Nguyen; Nguyen, Tat Thang; Trần, Vinh Cao; Tran, Minh–Triet; Ngo, Thanh Duc; Nguyen, Thien Huu; Hoai, Minh

doi:10.1109/cvpr46437.2021.00730

Cited by 53 publications

(21 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we experimentally validate our proposed TANGER by comparing the performance with the state-of-theart methods on several public datasets as well as one newly collected multilingual dataset TsiText. First, we examine the performance of TANGER for multilingual scene text recognition in comparison with two end-to-end methods [10,22] and one dictionary-guided method [34]. Then, we compare our model with the vision transformer ViTSTR [5] in three variants, i.e., tiny, small, and base versions for monolingual scene text recognition.…”

Section: Resultsmentioning

confidence: 99%

Section: B Performance On Multilingual Scene Textsmentioning

confidence: 99%

“…Table I lists the performance of the proposed TANGER with three state-of-the-art algorithms for multilingual scene text recognition, namely ABCNet+D [34], E2E-MLT [10], and Multiplexed [22]. ABCNet+D [34] is a dictionary-based recognition algorithm that incorporates dictionaries before handling ambiguous cases. E2E-MLT [10] is an end-to-end approach with a single fully convolutional network applicable to both Latin and non-Latin languages for multilingual scene text.…”

Section: B Performance On Multilingual Scene Textsmentioning

confidence: 99%

See 2 more Smart Citations

Augmented Transformers with Adaptive n-grams Embedding for Multilingual Scene Text Recognition

Yan¹,

Fang²,

Jin³

2023

Preprint

View full text Add to dashboard Cite

While vision transformers have been highly successful in improving the performance in image-based tasks, not much work has been reported on applying transformers to multilingual scene text recognition due to the complexities in the visual appearance of multilingual texts. To fill the gap, this paper proposes an augmented transformer architecture with ngrams embedding and cross-language rectification (TANGER). TANGER consists of a primary transformer with single patch embeddings of visual images, and a supplementary transformer with adaptive n-grams embeddings that aims to flexibly explore the potential correlations between neighbouring visual patches, which is essential for feature extraction from multilingual scene texts. Cross-language rectification is achieved with a loss function that takes into account both language identification and contextual coherence scoring. Extensive comparative studies are conducted on four widely used benchmark datasets as well as a new multilingual scene text dataset containing Indonesian, English, and Chinese collected from tourism scenes in Indonesia. Our experimental results demonstrate that TANGER is considerably better compared to the state-of-the-art, especially in handling complex multilingual scene texts.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: B Performance On Multilingual Scene Textsmentioning

confidence: 99%

Section: B Performance On Multilingual Scene Textsmentioning

confidence: 99%

See 1 more Smart Citation

Augmented Transformers with Adaptive n-grams Embedding for Multilingual Scene Text Recognition

Yan¹,

Fang²,

Jin³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Liu et al [48] perform text recognition using feature pyramids. We demonstrate that, without any task-specific engineering, we reconstruct fine details to perform robustly in dark, noisy conditions on SOTA text recognition methods [20,19,26,4,58], such as PARSeq [7].…”

Section: Related Workmentioning

confidence: 99%

Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition

Nguyen¹,

Chan²,

Bergman³

et al. 2023

Preprint

View full text Add to dashboard Cite

Images are indispensable for the automation of highlevel tasks, such as text recognition. Low-light conditions pose a challenge for these high-level perception stacks, which are often optimized on well-lit, artifact-free images. Reconstruction methods for low-light images can produce well-lit counterparts, but typically at the cost of highfrequency details critical for downstream tasks. We propose Diffusion in the Dark (DiD), a diffusion model for lowlight image reconstruction that provides qualitatively competitive reconstructions with that of SOTA, while preserving high-frequency details even in extremely noisy, dark conditions. We demonstrate that DiD, without any task-specific optimization, can outperform SOTA low-light methods in low-light text recognition on real images, bolstering the potential of diffusion models for ill-posed inverse problems.

show abstract

“…As a result, they continuously refine their visual module to obtain increasingly robust visual features and boost their recognition accuracy. Language-based methods 15 – 18 require an additional module to learn linguistic information that can assist with text prediction. This leads to a better performance than working without a language module.…”

Section: Introductionmentioning

confidence: 99%

Multilingual semantic fusion network for text recognition in the wild

et al. 2023

View full text Add to dashboard Cite

Most current approaches in the literature of scene text recognition train the language model via a text dataset far sparser than in natural language processing, resulting in inadequate training. Therefore, we propose a simple transformer encoder-decoder model called the multilingual semantic fusion network (MSFN) that can leverage prior linguistic knowledge to learn robust language features. First, we label the text dataset with forward, backward sequences, and subwords, which are extracted by tokenization with linguistic information. Then we introduce a multilingual model to the decoder corresponding to three different channels of the labeled dataset. The final output is fused by different channels to get more accurate results. In experiments, MSFN achieves cutting-edge performance across six benchmark datasets, and extensive ablative studies have proven the effectiveness of the proposed method. Code is available at https://github .com/lclee0577/MLViT.

show abstract

Dictionary-guided Scene Text Recognition

Cited by 53 publications

References 17 publications

Augmented Transformers with Adaptive n-grams Embedding for Multilingual Scene Text Recognition

Augmented Transformers with Adaptive n-grams Embedding for Multilingual Scene Text Recognition

Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition

Multilingual semantic fusion network for text recognition in the wild

Contact Info

Product

Resources

About