GTC: Guided Training of CTC Towards Efficient and Accurate Scene Text Recognition

Hu, Wenyang; Cai, Xiaocong; Hou, Jun; Yi, Shuai; Lin, Zhiping

doi:10.48550/arxiv.2002.01276

“…Moreover, inspired by SRN (Yu et al 2020) and GTC (Hu et al 2020), we propose a graph refinement module (GRM) to make secondary reasoning to improve the end-to-end performance further. The points in a text sequence are formulated as nodes in a graph, where the representation of each node is enhanced with semantic context and visual context information from its neighbors, and the character classification result should be more accurate.…”

Section: Introductionmentioning

confidence: 99%

PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network

Wang

¹

,

Zhang

²

,

Qi

³

et al. 2021

AAAI

View full text Add to dashboard Cite

The reading of arbitrarily-shaped text has received increasing research attention. However, existing text spotters are mostly built on two-stage frameworks or character-based methods, which suffer from either Non-Maximum Suppression (NMS), Region-of-Interest (RoI) operations, or character-level annotations. In this paper, to address the above problems, we propose a novel fully convolutional Point Gathering Network (PGNet) for reading arbitrarily-shaped text in real-time. The PGNet is a single-shot text spotter, where the pixel-level character classification map is learned with proposed PG-CTC loss avoiding the usage of character-level annotations. With a PG-CTC decoder, we gather high-level character classification vectors from two-dimensional space and decode them into text symbols without NMS and RoI operations involved, which guarantees high efficiency. Additionally, reasoning the relations between each character and its neighbors, a graph refinement module (GRM) is proposed to optimize the coarse recognition and improve the end-to-end performance. Experiments prove that the proposed method achieves competitive accuracy, meanwhile significantly improving the running speed. In particular, in Total-Text, it runs at 46.7 FPS, surpassing the previous spotters with a large margin.

show abstract

“…Recently, the pursuit of high performance on benchmarks has drawn much attention from the community. Driven by deep learning [50,31,2,33,12] and large volume of synthetic data [13,29,46], the recognition accuracy on standard benchmarks has escalated rapidly. For instance, the accuracy on IIIT-5k [27] without lexicon has increased from 78.2% [31] to 96.0% [12] in a very short period.…”

Section: Introductionmentioning

confidence: 99%

On Vocabulary Reliance in Scene Text Recognition

Wan

¹

,

Zhang

²

,

Zhang

³

et al. 2020

Preprint

0

View full text Add to dashboard Cite

The pursuit of high performance on public benchmarks has been the driving force for research in scene text recognition, and notable progress has been achieved. However, a close investigation reveals a startling fact that the state-ofthe-art methods perform well on images with words within vocabulary but generalize poorly to images with words outside vocabulary. We call this phenomenon "vocabulary reliance". In this paper, we establish an analytical framework to conduct an in-depth study on the problem of vocabulary reliance in scene text recognition. Key findings include: (1) Vocabulary reliance is ubiquitous, i.e., all existing algorithms more or less exhibit such characteristic;(2) Attention-based decoders prove weak in generalizing to words outside vocabulary and segmentation-based decoders perform well in utilizing visual features; (3) Context modeling is highly coupled with the prediction layers. These findings provide new insights and can benefit future research in scene text recognition. Furthermore, we propose a simple yet effective mutual learning strategy to allow models of two families (attention-based and segmentationbased) to learn collaboratively. This remedy alleviates the problem of vocabulary reliance and improves the overall scene text recognition performance.

show abstract

PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network

Wang¹,

Zhang²,

Qi³

et al. 2021

Preprint

View full text Add to dashboard Cite

The reading of arbitrarily-shaped text has received increasing research attention. However, existing text spotters are mostly built on two-stage frameworks or character-based methods, which suffer from either Non-Maximum Suppression (NMS), Region-of-Interest (RoI) operations, or character-level annotations. In this paper, to address the above problems, we propose a novel fully convolutional Point Gathering Network (PGNet) for reading arbitrarily-shaped text in real-time. The PGNet is a single-shot text spotter, where the pixel-level character classification map is learned with proposed PG-CTC loss avoiding the usage of character-level annotations. With a PG-CTC decoder, we gather high-level character classification vectors from two-dimensional space and decode them into text symbols without NMS and RoI operations involved, which guarantees high efficiency. Additionally, reasoning the relations between each character and its neighbors, a graph refinement module (GRM) is proposed to optimize the coarse recognition and improve the end-to-end performance. Experiments prove that the proposed method achieves competitive accuracy, meanwhile significantly improving the running speed. In particular, in Total-Text, it runs at 46.7 FPS, surpassing the previous spotters with a large margin.

show abstract

GTC: Guided Training of CTC Towards Efficient and Accurate Scene Text Recognition

Cited by 3 publications

References 24 publications

PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network

PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network

On Vocabulary Reliance in Scene Text Recognition

PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network

Contact Info

Product

Resources

About