SynthText3D: synthesizing scene text images from 3D virtual worlds

Liao, Minghui; Song, Boyu; He, Minghang; Yao, Cong; Bai, Xiang

doi:10.1007/s11432-019-2737-0

Cited by 40 publications

(18 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method aims at use cases involving creative self expression and augmented reality (e.g., photo-realistic translation, leveraging multi-lingual OCR technologies [59]). Our method can be used for data generation and augmentation for training future OCR systems, as successfully done by others [49], [60] and in other domains [61], [62]. We are aware, however, that like other technologies, ours can be misused, possibly the same as deepfake faces can be used for misinformation.…”

Section: Discussionmentioning

confidence: 87%

TextStyleBrush: Transfer of Text Aesthetics from a Single Example

Krishnan¹,

Kovvuri²,

Pang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a novel approach for disentangling the content of a text image from all aspects of its appearance. The appearance representation we derive can then be applied to new content, for one-shot transfer of the source style to new content. We learn this disentanglement in a self-supervised manner. Our method processes entire word boxes, without requiring segmentation of text from background, per-character processing, or making assumptions on string lengths. We show results in different text domains which were previously handled by specialized methods, e.g., scene text, handwritten text. To these ends, we make a number of technical contributions: (1) We disentangle the style and content of a textual image into a non-parametric, fixed-dimensional vector. (2) We propose a novel approach inspired by StyleGAN but conditioned over the example style at different resolution and content. (3) We present novel self-supervised training criteria which preserve both source style and target content using a pre-trained font classifier and text recognizer. Finally, (4) we also introduce Imgur5K, a new challenging dataset for handwritten word images. We offer numerous qualitative photo-realistic results of our method. We further show that our method surpasses previous work in quantitative tests on scene text and handwriting datasets, as well as in a user study.

show abstract

Section: Discussionmentioning

confidence: 87%

TextStyleBrush: Transfer of Text Aesthetics from a Single Example

Krishnan¹,

Kovvuri²,

Pang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In STR, it has become a standard practice to use synthetic datasets. We introduce previous studies providing synthetic data generation algorithms for STR [5] and other tasks that can be exploited for STR [2,8,12].…”

Section: Related Workmentioning

confidence: 99%

“…However, the use of off-the-shelf segmentation techniques for text-background alignment can produce erroneous predictions and result in unrealistic text images. Recent studies like SynthText3D [8] and UnrealText [12] address this problem by synthesizing images with 3D graphic engines. Experiment results show that text detection performance can be notably improved by using synthesized text images without text alignment error.…”

Section: Related Workmentioning

confidence: 99%

SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Models

Yim¹,

Kim²,

Cho³

et al. 2021

Preprint

View full text Add to dashboard Cite

For successful scene text recognition (STR) models, synthetic text image generators have alleviated the lack of annotated text images from the real world. Specifically, they generate multiple text images with diverse backgrounds, font styles, and text shapes and enable STR models to learn visual patterns that might not be accessible from manually annotated data. In this paper, we introduce a new synthetic text image generator, SynthTIGER, by analyzing techniques used for text image synthesis and integrating effective ones under a single algorithm. Moreover, we propose two techniques that alleviate the long-tail problem in length and character distributions of training data. In our experiments, SynthTIGER achieves better STR performance than the combination of synthetic datasets, MJSynth (MJ) and Synth-Text (ST). Our ablation study demonstrates the benefits of using subcomponents of SynthTIGER and the guideline on generating synthetic text images for STR models. Our implementation is publicly available at https://github.com/clovaai/synthtiger.

show abstract

“…As a result of the rapid development of computer vision [2,18,24,28,29], methods based on the GAN model have been widely explored for text-to-image synthesis tasks. Reed et al [12] made initial attempts to conduct text-to-image synthesis with the conditional DCGAN model.…”

Section: Text-to-image Synthesismentioning

confidence: 99%

Learning efficient text-to-image synthesis via interstage cross-sample similarity distillation

Mao

Chang

et al. 2020

Sci. China Inf. Sci.

View full text Add to dashboard Cite

For a given text, previous text-to-image synthesis methods commonly utilize a multistage generation model to produce images with high resolution in a coarse-to-fine manner. However, these methods ignore the interaction among stages, and they do not constrain the consistent cross-sample relations of images generated in different stages. These deficiencies result in inefficient generation and discrimination. In this study, we propose an interstage cross-sample similarity distillation model based on a generative adversarial network (GAN) for learning efficient text-to-image synthesis. To strengthen the interaction among stages, we achieve interstage knowledge distillation from the refined stage to the coarse stages with novel interstage cross-sample similarity distillation blocks. To enhance the constraint on the cross-sample relations of the images generated at different stages, we conduct cross-sample similarity distillation among the stages. Extensive experiments on the Oxford-102 and Caltech-UCSD Birds-200-2011 (CUB) datasets show that our model generates visually pleasing images and achieves quantitatively comparable performance with state-of-the-art methods.

show abstract

SynthText3D: synthesizing scene text images from 3D virtual worlds

Cited by 40 publications

References 44 publications

TextStyleBrush: Transfer of Text Aesthetics from a Single Example

TextStyleBrush: Transfer of Text Aesthetics from a Single Example

SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Models

Learning efficient text-to-image synthesis via interstage cross-sample similarity distillation

Contact Info

Product

Resources

About