A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution

Ma, Jianqi; Liang, Zhetong; Zhang, Lei

doi:10.1109/cvpr52688.2022.00582

Cited by 54 publications

(16 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[32] proposes a content perceptual loss to extract multi-scale text recognition features to conduct a content aware supervision. TPGSR [12], TATT [13], and C3-STISR [14] extract text-specific clues to guide the superresolution. In particular, TPGSR is the first method that additionally introduces a scene text recognizer to provide text priors.…”

Section: B Scene Text Image Super-resolutionmentioning

confidence: 99%

“…Then, the extracted priors are fed into the superresolution to iteratively benefit the super-resolution. TATT [13] introduces a transformer-based module, which leverages global attention mechanism, to exert the semantic guidance of text prior to the text reconstruction process. C3-STISR [14] is proposed to learn triple clues, including recognition clue from a STR, linguistical clue from a language model, and a visual clue from a skeleton painter to rich the representation of the text-specific clue.…”

Section: B Scene Text Image Super-resolutionmentioning

confidence: 99%

“…[11] and TG [9] extract stroke-level information from HR images through specific networks to provide more fine-grained supervision information. [12], [13], [14] additionally introduce external modules to extract various textspecific clues to facilitate the recovery and use the supervision from HR images to finetune their modules.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Positional Information is a Strong Supervision for Volumetric Medical Image Segmentation

Yu-hong

Hou

Zeng

et al. 2023

J. Shanghai Jiaotong Univ. (Sci.)

View full text Add to dashboard Cite

Scene text image super-resolution (STISR) is an important pre-processing technique for text recognition from lowresolution scene images. Nowadays, various methods have been proposed to extract text-specific information from high-resolution (HR) images to supervise STISR model training. However, due to uncontrollable factors (e.g. shooting equipment, focus, and environment) in manually photographing HR images, the quality of HR images cannot be guaranteed, which unavoidably impacts STISR performance. Observing the quality issue of HR images, in this paper we propose a novel idea to boost STISR by first enhancing the quality of HR images and then using the enhanced HR images as supervision to do STISR. Concretely, we develop a new STISR framework, called High-Resolution ENhancement (HiREN) that consists of two branches and a quality estimation module. The first branch is developed to recover the low-resolution (LR) images, and the other is an HR quality enhancement branch aiming at generating high-quality (HQ) text images based on the HR images to provide more accurate supervision to the LR images. As the degradation from HQ to HR may be diverse, and there is no pixel-level supervision for HQ image generation, we design a kernel-guided enhancement network to handle various degradation, and exploit the feedback from a recognizer and text-level annotations as weak supervision signal to train the HR enhancement branch. Then, a quality estimation module is employed to evaluate the qualities of HQ images, which are used to suppress the erroneous supervision information by weighting the loss of each image. Extensive experiments on TextZoom show that HiREN can work well with most existing STISR methods and significantly boost their performances.

show abstract

Section: B Scene Text Image Super-resolutionmentioning

confidence: 99%

Section: B Scene Text Image Super-resolutionmentioning

confidence: 99%

See 1 more Smart Citation

Positional Information is a Strong Supervision for Volumetric Medical Image Segmentation

Yu-hong

Hou

Zeng

et al. 2023

J. Shanghai Jiaotong Univ. (Sci.)

View full text Add to dashboard Cite

show abstract

“…The research on scene text image super-resolution focuses on the recognizability of text images, and due to the variability and complexity of the text presentation form, there are still many problems and challenges in the current research. The Sequential Residual Block of some network models (Ma et al, 2022;Wang et al, 2020) consist of recurrent neural networks, which mainly use position-based operations and cannot effectively capture the long-distance correlation in text images, and performance degradation occurs when dealing with long text (Qin et al, 2022). Some scene text images have problems such as image occlusion and content missing, so using binarized masks as image inputs could not complement the semantic information of text images, resulting in poor processing results of such images.…”

Section: Introductionmentioning

confidence: 99%

Fusing Self-Attention and Scene Prior for Text Super-Resolution

Zhou,

Yu,

et al. 2024

Preprint

View full text Add to dashboard Cite

In urban scenes, street scene text images often contain rich semantic information, which is a key cue for perceiving and understanding the scene. Recovering high-resolution text image from low-resolution text image is a challenging task due to the complex background of street scene text images, irregular text shapes, image blurring, distortion and deformation. Existing approaches mainly use recurrent neural networks to mine text-specific contextual information, which cannot effectively capture long-range correlations in text images and fail to effectively utilize the semantic information of text images. In addition, they tend to have weak generalization when dealing with cross-linguistics. Aiming at the above problems, we propose a text-image super-resolution method that fuses self-attention and scene prior. Firstly, a text recognition network and a semantic segmentation network are used to extract prior features, which are fused with the image visual features through a prior interpreter to realize the effective utilization of text semantic and visual information. Then the recursive information in the text line is extracted by Transformer, and the sequence information is extracted by using the global visibility of multi-head attention to establish the correlation between the front and back characters, to solve the problem of performance degradation when the model is processing long text. Finally, the ability of the model to extract text contour and process deformed text is enhanced by gradient profile loss and text structure-aware loss. The experimental results show that the recognition accuracy of our method in TextZoom's three types of test sets is higher than that of the baseline model TSRN 4%, and the average values of the metrics in PSNR and SSIM reach 21.34 and 78.39, which effectively improves the application effect of the text super-resolution model in real scenes.

show abstract

“…In recent years, many prior-guided deep learning-based approaches have attracted much attention in the field of STISR, which can be roughly divided into three major categories: that is, semantic priorguided methods [1][2][3], structure prior-guided methods [5,9,10], and prior-hybrid methods [6].…”

mentioning

confidence: 99%

SECANet: A structure‐enhanced attention network with dual‐domain contrastive learning for scene text image super‐resolution

He,

Zhang,

Zhang

et al. 2023

Electronics Letters

View full text Add to dashboard Cite

In this letter, we developed novel Structure Enhanced Channel Attention Network (SECANet) for scene text image super‐resolution (STISR). The newly proposed SECANet integrates a group of Structure‐Enhanced Attention Modules to focus more on both local and global structural features in the character regions of text images. Moreover, we elaborately formulate a Dual‐Domain Contrastive Learning framework that integrates one pixel‐level contrastive loss and the other semantic‐level contrastive loss to jointly optimize the SECANet for generating more visually pleasing yet better recognizable high‐quality SR images without introducing any additional prior generators in both the training and testing stages, showing promising computational efficiency. Experimental results on the Textzoom dataset indicate that our method can achieve both decent performance in super‐resolving more impressive scene text images from low‐resolution ones and better recognition accuracy than other competitors.

show abstract

A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution

Cited by 54 publications

References 32 publications

Positional Information is a Strong Supervision for Volumetric Medical Image Segmentation

Positional Information is a Strong Supervision for Volumetric Medical Image Segmentation

Fusing Self-Attention and Scene Prior for Text Super-Resolution

SECANet: A structure‐enhanced attention network with dual‐domain contrastive learning for scene text image super‐resolution

Contact Info

Product

Resources

About