MTRNet++: One-stage mask-based scene text eraser

Tursun, Osman; Denman, Simon; Zeng, Rui; Sivapalan, Sabesan; Sridharan, Sridha; Fookes, Clinton

doi:10.1016/j.cviu.2020.103066

Cited by 37 publications

(20 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evaluation is performed on 256 × 256 images. Outputs for MTRNet [3], MTRNet++ [4] and EnsNet [2] were provided by the authors of MTRNet++. For our experiments, we reimplement the ResNet-50 variant of WS-TE [5] and train the inpainting model on the Places2 dataset [21].…”

Section: Resultsmentioning

confidence: 99%

“…Hence, we'd like to penalize the false negatives more than false positives. To achieve this, we train our text segmentation network using the Tversky Loss [9,4], which is a generalization of the Dice Loss.…”

Section: Text Mask Generatormentioning

confidence: 99%

“…However, their method divides images into patches, which often produces artifacts at patch borders and fails to erase large characters due to the lack of global context. EnsNet [2] and MTRNets [3,4] process entire images with GANs, removing border artifacts, but struggle to identify text in real-life images, leaving obvious artifacts in the results. WS-TE [5] propose a two-stage weak supervision approach which contains a text segmentation model to mask text-instances and an inpainting model to fill in the masked pixels.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Two-Stage Seamless Text Erasing On Real-World Scene Images

Conrad¹,

Chen²

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Erasing text from images is a common image-editing task in film industry and shared media. Existing text-erasing models either tend to produce artifacts or fail to remove all the text in real-world images. In this paper, we follow a two-stage text erasing framework that first masks the text by segmentation, and then inpaints the masked region to create a text-erased image. Our proposed text mask generator is designed to accurately cover text, which combined with inpainting, can produce reliable text-erased results. In the inpainting model, we propose a Multiscale Gradient Reconstruction Loss to generate sharp realistic-looking images. Our model achieves stateof-the-art results on both synthetic and real world data in both quantitative and qualitative measures.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Text Mask Generatormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Two-Stage Seamless Text Erasing On Real-World Scene Images

Conrad¹,

Chen²

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…Liu et al [4] provided a comprehensive real-world scenetext removal benchmark, named SCUT-EnsText, and proposed EraseNet adopting a coarse-to-fine erasure network structure with a segmentation head, which can generate a mask of the text region to help with text region localization. MTRNet++ [2] shares the same coarse-to-fine inpainting idea but uses a multi-branch generator. The mask-refine branch could predict stroke-level text masks to guide text removal.…”

Section: Text Erasingmentioning

confidence: 99%

“…After the prior work of Scene Text Eraser [1], scene text erasing research has developed into two directions: one-step and two-step methods. [2]- [4] are the representative works of one-step methods combining the text detection and inpainting functions into one network, which makes one-step method Manuscript received March 10, 2021. This work was partially supported by JSPS KAKENHI Grant Numbers 18K19772, 19K12033, 20H04201.…”

Section: Introductionmentioning

confidence: 99%

Stroke-Based Scene Text Erasing Using Synthetic Data for Training

Tang,

Miyazaki,

Sugaya

et al. 2021

Preprint

View full text Add to dashboard Cite

Scene text erasing, which replaces text regions with reasonable content in natural images, has drawn attention in the computer vision community in recent years. There are two potential subtasks in scene text erasing: text detection and image inpainting. Either sub-task requires considerable data to achieve better performance; however, the lack of a large-scale real-world scene-text removal dataset allows the existing methods to not work in full strength. To avoid the limitation of the lack of pairwise real-world data, we enhance and make full use of the synthetic text and consequently train our model only on the dataset generated by the improved synthetic text engine. Our proposed network contains a stroke mask prediction module and background inpainting module that can extract the text stroke as a relatively small hole from the text image patch to maintain more background content for better inpainting results. This model can partially erase text instances in a scene image with a bounding box provided or work with an existing scene text detector for automatic scene text erasing. The experimental results of qualitative evaluation and quantitative evaluation on the SCUT-Syn, ICDAR2013, and SCUT-EnsText datasets demonstrate that our method significantly outperforms existing state-of-the-art methods even when trained on real-world data.

show abstract

Mining text from natural scene and video images: A survey

Shivakumara

Alaei

Pal

2021

WIREs Data Min & Knowl

View full text Add to dashboard Cite

In computer terminology, mining is considered as extracting meaningful information or knowledge from a large amount of data/information using computers. The meaningful information can be extracted from normal text, and images obtained from different resources, such as natural scene images, video, and documents by deriving semantics from text and content of the images. Although there are many pieces of work on text/data mining and several survey/review papers are published in the literature, to the best of our knowledge there is no survey paper on mining textual information from the natural scene, video, and document images considering word spotting techniques. In this article, we, therefore, provide a comprehensive review of both the non‐spotting and spotting based mining techniques. The mining approaches are categorized as feature, learning and hybrid‐based methods to analyze the strengths and limitations of the models of each category. In addition, it also discusses the usefulness of the methods according to different situations and applications. Furthermore, based on the review of different mining approaches, this article identifies the limitations of the existing methods and suggests new applications and future directions to continue the research in multiple directions. We believe such a review article will be useful to the researchers to quickly become familiar with the state‐of‐the‐art information and progresses made toward mining textual information from natural scene and video images. This article is categorized under: Algorithmic Development > Text Mining

show abstract

MTRNet++: One-stage mask-based scene text eraser

Cited by 37 publications

References 10 publications

Two-Stage Seamless Text Erasing On Real-World Scene Images

Two-Stage Seamless Text Erasing On Real-World Scene Images

Stroke-Based Scene Text Erasing Using Synthetic Data for Training

Mining text from natural scene and video images: A survey

Contact Info

Product

Resources

About