Reading scene text with fully convolutional sequence modeling

Gao, Yunze; Chen, Yingying; Wang, Jinqiao; Tang, Ming; Lu, Hanqing

doi:10.1016/j.neucom.2019.01.094

Cited by 79 publications

(51 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among them, the first group contains several well-known recognition networks, including CRNN [1] and GRCNN [5]. We then compare ours with previous attention aware approaches such as FAN [8], FCN [14], and Baek et al [16].…”

Section: Configurationmentioning

confidence: 99%

Scene Text Recognition with Temporal Convolutional Encoder

Zheng³

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Texts from scene images typically consist of several characters and exhibit a characteristic sequence structure. Existing methods capture the structure with the sequence-tosequence models by an encoder to have the visual representations and then a decoder to translate the features into the label sequence. In this paper, we study text recognition framework by considering the long-term temporal dependencies in the encoder stage. We demonstrate that the proposed Temporal Convolutional Encoder with increased sequential extents improves the accuracy of text recognition. We also study the impact of different attention modules in convolutional blocks for learning accurate text representations. We conduct comparisons on seven datasets and the experiments demonstrate the effectiveness of our proposed approach.

show abstract

Section: Configurationmentioning

confidence: 99%

Scene Text Recognition with Temporal Convolutional Encoder

Zheng³

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Lee et al [17] combined a recursive CNN with a recurrent CNN in their R 2 AM to capture long-term dependencies when extracting features from raw images, and then fed these features to an attention-RNN network for sequential transcription. Gao et al [4,12] designed two models to compare the performance of CNN and LSTM in terms of sequential feature encoding. According to their experiments, features extracted by LSTM were more powerful than those extracted by CNN.…”

Section: Related Workmentioning

confidence: 99%

“…However, with the development of LSTM-based recognizers, the DCNN ones were quickly and significantly surpassed. Recently, some researchers argued that LSTM-based models were hard to train [12] and not able to achieve good performance on non-horizontal text [10], so explorations on models without LSTM started again. For instance, STN-OCR [9] utilized fully connected layers and a fixed number of softmax classifiers for sequential prediction; SqueezedText [20] employed a binary convolutional encoder-decoder network to generate salience maps for individual characters and then exploited a GRU-based bi-RNN for further correction; Liao et al [10] proposed to address the scene text recognition issue from a 2-D perspective with a CA-FCN model, so that the spatial information could be taken into account when performing prediction.…”

Section: Related Workmentioning

confidence: 99%

“…In other recognizers [4,10,12], the feature maps F and maps A generated for other purposes are always combined with the element-wise multiplication ⊗ in the way of F out = F ⊗ (1 + A). However, in our experiments we find that directly concatenating feature maps F and character center masks M achieves better performance, which means the subsequent attention-based module prefers to learn patterns from F and M directly, rather than from their fused results.…”

Section: Focused Attention Modulementioning

confidence: 99%

“…However, in existing models, both FC-LSTM and ConvLSTM are used only for frame-level prediction and are incapable of producing sequential outputs from one single input image unless the Connectionist Temporal Classification (CTC) [4,7,12] or attention mechanism [5,6,8,13] is incorporated. To perform sequential prediction and, meanwhile, provide the model spatial awareness, we further improve ConvL-STM by embedding the attention mechanism into the structure.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

FACLSTM: ConvLSTM with focused attention for scene text recognition

Wang¹,

Jia²,

He³

et al. 2020

Sci. China Inf. Sci.

View full text Add to dashboard Cite

Scene text recognition has recently been widely treated as a sequence-to-sequence prediction problem, where traditional fully-connected-LSTM (FC-LSTM) has played a critical role. Due to the limitation of FC-LSTM, existing methods have to convert 2-D feature maps into 1-D sequential feature vectors, resulting in severe damages of the valuable spatial and structural information of text images. In this paper, we argue that scene text recognition is essentially a spatiotemporal prediction problem for its 2-D image inputs, and propose a convolution LSTM (ConvLSTM)-based scene text recognizer, namely, FACLSTM, i.e., Focused Attention ConvLSTM, where the spatial correlation of pixels is fully leveraged when performing sequential prediction with LSTM. Particularly, the attention mechanism is properly incorporated into an efficient ConvLSTM structure via the convolutional operations and additional character center masks are generated to help focus attention on right feature areas. The experimental results on benchmark datasets IIIT5K, SVT and CUTE demonstrate that our proposed FACLSTM performs competitively on the regular, low-resolution and noisy text images, and outperforms the state-of-the-art approaches on the curved text images with large margins.

show abstract

From object detection to text detection and recognition: A brief evolution history of optical character recognition

Wang

Pan

Guo

et al. 2021

WIREs Computational Stats

View full text Add to dashboard Cite

Text detection and recognition, which is also known as optical character recognition (OCR), is an active research area under quick development with a lot of exciting applications. Deep‐learning‐based methods represent the state‐of‐art of this area. However, these methods are largely deterministic: they give a deterministic output for each input. For both statisticians and general users, methods supporting uncertainty inference are of great appeal, leaving rich research opportunities to incorporate statistical models and methods with the established deep‐learning‐based approaches. In this paper, we provide a comprehensive review of the evolution history of research development on OCR with discussions on the statistical insights behind these developments and potential directions to enhance the current methods with statistical approaches. We hope this article can serve as a useful guidebook for statisticians who are seeking for a path toward edge‐cutting research in this exciting area. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Deep Learning Data: Types and Structure > Image and Spatial Data

show abstract

Reading scene text with fully convolutional sequence modeling

Cited by 79 publications

References 28 publications

Scene Text Recognition with Temporal Convolutional Encoder

Scene Text Recognition with Temporal Convolutional Encoder

FACLSTM: ConvLSTM with focused attention for scene text recognition

From object detection to text detection and recognition: A brief evolution history of optical character recognition

Contact Info

Product

Resources

About