Editing Text in the Wild

Abstract

In this paper, we are interested in editing text in natural images, which aims to replace or modify a word in the source image with another one while maintaining its realistic look. This task is challenging, as the styles of both background and text need to be preserved so that the edited image is visually indistinguishable from the source image. Specifically, we propose an end-to-end trainable style retention network (SRNet) that consists of three modules: text conversion module, background inpainting module and fusion module. The text conversion module changes the text content of the source image into the target text while keeping the original text style. The background inpainting module erases the original text, and fills the text region with appropriate texture. The fusion module combines the information from the two former modules, and generates the edited text images. To our knowledge, this work is the first attempt to edit text in natural images at the word level. Both visual effects and quantitative results on synthetic and real-world dataset (ICDAR 2013) fully confirm the importance and necessity of modular decomposition. We also conduct extensive experiments to validate the usefulness of our method in various real-world applications such as text image synthesis, augmented reality (AR) translation, information hiding, etc.

Realistic Text Replacement With Non-Uniform Style Conditioning

“…Our best model is currently capable of reaching a per image accuracy of 0.697, which is comparable to the accuracy of 0.824 achieved by the model on raw data. We outperform the work of [1], which achieves a per image accuracy of 0.517. The values of the validation proxy metrics could be seen in Table 3.…”

“…The task of the fusion module is to fuse the conversion module results and the background inpainting module results. The key difference is that [1] uses a text skeleton based conversion module, while [24] uses a control points based one. Also the modules of [1] are learned independently while [24] proposes a fully differentiable architecture.…”

“…Unlike earlier works [1], [25], we design an architecture capable of text replacement of several text areas in one forward pass. To accomplish this, we introduce a novel nonuniform conditional normalization layer.…”

“…In order to compute the output of the PatchAdaIn layer we iterate through all the patches and extract mean and standard deviation statistics of the areas denoted by the patches from the style features 1 . Both statistics are calculated across the width and height dimension.…”

“…T HE task of realistic text replacement [1] could be formulated as follows: replace text present in an image with arbitrary user-supplied text in a way that will not allow distinguishing the resulting image from the original one. One of the technologies which stand to primarily benefit from realistic text replacement is augmented reality (AR) text translation.…”