Gaussian Constrained Attention Network for Scene Text Recognition

Qiao, Zhiqiang; Qin, Xugong; Zhou, Yu; Yang, Fei; Wang, Weiping

doi:10.1109/icpr48806.2021.9412806

Cited by 20 publications

(5 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance on five public benchmarks demonstrates the effectiveness and robustness of our approach. We'd like to combine text recognition [39,40], self-supervised learning [14,31,32,69,75,76], and knowledge distillation [65,66] to build a robust text reading system.…”

Section: Discussionmentioning

confidence: 99%

Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection

Qin¹,

Zhou²,

Guo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Due to the large success in object detection and instance segmentation, Mask R-CNN attracts great attention and is widely adopted as a strong baseline for arbitrary-shaped scene text detection and spotting. However, two issues remain to be settled. The first is dense text case, which is easy to be neglected but quite practical. There may exist multiple instances in one proposal, which makes it difficult for the mask head to distinguish different instances and degrades the performance. In this work, we argue that the performance degradation results from the learning confusion issue in the mask head. We propose to use an MLP decoder instead of the "deconv-conv" decoder in the mask head, which alleviates the issue and promotes robustness significantly. And we propose instanceaware mask learning in which the mask head learns to predict the shape of the whole instance rather than classify each pixel to text or non-text. With instance-aware mask learning, the mask branch can learn separated and compact masks. The second is that due to large variations in scale and aspect ratio, RPN needs complicated anchor settings, making it hard to maintain and transfer across different datasets. To settle this issue, we propose an adaptive label assignment in which all instances especially those with extreme aspect ratios are guaranteed to be associated with enough anchors. Equipped with these components, the proposed method named MAYOR 1 achieves state-of-the-art performance on five benchmarks including DAST1500, MSRA-TD500, ICDAR2015, CTW1500, and Total-Text.

show abstract

Section: Discussionmentioning

confidence: 99%

Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection

Qin¹,

Zhou²,

Guo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Then, building polygon prediction module outputs the hidden state h t from the previous input. Furthermore, an attention module with Gaussian constraints [45] calculates the attention weight α t to integrate building feature B and hidden state h t , which can better focus on the local region of the previously predicted vertices y t−2 and y t−1 . Afterward, the region-related coefficient is calculated from attention weight α t and building feature B using an element-wise product.…”

Section: Bidirectional Building Polygon Prediction Modulementioning

confidence: 99%

Topological Building Extraction With Bidirectional Prediction From Remote Sensing Images

Zhang,

Du,

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Topological building extraction in remote sensing images is vital for city planning, disaster assessment, and other real-world applications. To meet the requirements of real-world applications, existing building extraction approaches predict topological building by vectorization of binary building masks using multiple refinement stages, leading to complex methodology and poor generalization. To tackle this issue, we propose a topological building extraction approach by directly predicting serialized vertices of each building instance. We observe that the order of serialized vertices from one building is inherently bidirectional, which can be clockwise or counterclockwise. By this new observation, the proposed method learns serialized vertices for each building supervised by the bidirectional constraint. Moreover, we design a cross-scale feature fusion module to obtain building representations with rich spatial and context information, facilitating the following serialized vertex prediction. Besides, a merge strategy is adopted to generate the final topological building from serialized vertices of two directions (clockwise and counterclockwise). Experiments are conducted on three building benchmarks to evaluate the effectiveness of our proposed method. Finally, extensive results show that the proposed approach outperforms state-of-the-art methods highlighting its superiority.

show abstract

“…[34] trains a deep BiLSTM encoder to extract broader contextual dependencies. To solve the problem of attention diffusion, [45] proposes a Gaussian constrained refinement module to refine the attention distribution. [86] adopts an external language model to introduce useful context information into the decoding.…”

Section: Autoregressive Text Recognitionmentioning

confidence: 99%

PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition

Qiao

Zhou

Wei

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Nowadays, scene text recognition has attracted more and more attention due to its various applications. Most state-of-the-art methods adopt an encoder-decoder framework with attention mechanism, which generates text autoregressively from left to right. Despite the convincing performance, the speed is limited because of the one-by-one decoding strategy. As opposed to autoregressive models, non-autoregressive models predict the results in parallel with a much shorter inference time, but the accuracy falls behind the autoregressive counterpart considerably. In this paper, we propose a Parallel, Iterative and Mimicking Network (PIMNet) to balance accuracy and efficiency. Specifically, PIMNet adopts a parallel attention mechanism to predict the text faster and an iterative generation mechanism to make the predictions more accurate. In each iteration, the context information is fully explored. To improve learning of the hidden layer, we exploit the mimicking learning in the training phase, where an additional autoregressive decoder is adopted and the parallel decoder mimics the autoregressive decoder with fitting outputs of the hidden layer. With the shared backbone between the two decoders, the proposed PIMNet can be trained end-to-end without pre-training. During inference, the branch of the autoregressive decoder is removed for a faster speed. Extensive experiments on public benchmarks demonstrate the effectiveness and efficiency of PIMNet. Our code is available in the supplementary material. CCS CONCEPTS• Applied computing → Optical character recognition.

show abstract

Gaussian Constrained Attention Network for Scene Text Recognition

Cited by 20 publications

References 51 publications

Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection

Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection

Topological Building Extraction With Bidirectional Prediction From Remote Sensing Images

PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition

Contact Info

Product

Resources

About