Crosslingual Handwritten Text Generation Using GANs

Chang, Chun Chieh; García, Paola; Khudanpur, Sanjeev

doi:10.1007/978-3-031-41501-2_20

Cited by 3 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The "Arabic Handwriting Data Base" (AHDB) [68], used in the studies conducted by [56,57], consists of characters and words derived from numerical values and bank check filling. Finally, the "Multilingual Automatic Document Classification Analysis and Translation" (MADCAT) database [69][70][71] was used by the work [51], and consists of handwritten Arabic documents scanned at high resolution, totaling 750,000 images of segmented lines.…”

Section: Recognition Tasks and Datasetsmentioning

confidence: 99%

“…For Vietnamese language, a small dataset was introduced in the Cinnamon AI Marathon competition: "the Cinnamon Handwritten OCR for Vietnamese Address Challenge" [75], which contains handwritten address images and was utilized in the work of [48]. Another Vietnamese dataset is the "Vietnamese Online Handwritten Text Recognition" (VNonDB) [76], used by [51], which is an online handwritten Vietnamese dataset released as a challenge for ICFHR2018 and converted into an offline version, comprising 100,000 images of word-level segmentations. In the case of Latin, a subset of the Bullinger dataset [47] was used in the work of [47].…”

Section: Recognition Tasks and Datasetsmentioning

confidence: 99%

“…Chang et al [51] presented a method using GANs to generate handwritten content across different languages, with the goal of enhancing handwriting recognition in low-resource contexts. They reported FID scores for the VNonDB Vietnamese dataset as 27.46, 77.10, and 142.08 for printed, crosslingual, and semi-supervised GANs, respectively.…”

Section: Deep Learningmentioning

confidence: 99%

See 2 more Smart Citations

Data Augmentation for Offline Handwritten Text Recognition: A Systematic Literature Review

de Sousa Neto,

Bezerra,

de Moura

et al. 2024

SN COMPUT. SCI.

View full text Add to dashboard Cite

Offline Handwritten Text Recognition (HTR) systems concern the automatic recognition and transcription of handwritten text from scanned images to digital media. Recently, HTR research field has become increasingly important due to the growing need for digitizing documents and automating data entry across various industries. However, achieving satisfactory results depend on the amount of available samples to train an optical model. Creating and labeling large enough datasets for this purpose often require significant time and effort, that in some situations may be impractical. To address this problem, data augmentation approaches are commonly used as an essential component of HTR systems. In this way, the present work aims to identify, explore, and analyze the scope of data augmentation approaches for offline HTR systems. Furthermore, we detailed our research protocol and answered four pertinent research questions, which enabled us to discuss trends and possible gaps. A search was conducted across five scientific databases, focusing on papers published between 2012 and 2023. The search yielded 976 primary papers, with 32 meeting the criteria for inclusion in this review. Our results indicate that handwriting synthesis is an emerging research field, and we observed that Digital Image Processing (DIP) is still widely used as an image generator. Nevertheless, the application of Generative Adversarial Networks (GAN) has gained traction in recent years owing to its impressive ability to synthesize images of handwritten text with arbitrary style and content. In addition, we explored and analyzed the most commonly used datasets and text recognition levels in the selected works.

show abstract

Section: Recognition Tasks and Datasetsmentioning

confidence: 99%

Section: Recognition Tasks and Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Data Augmentation for Offline Handwritten Text Recognition: A Systematic Literature Review

de Sousa Neto,

Bezerra,

de Moura

et al. 2024

SN COMPUT. SCI.

View full text Add to dashboard Cite

show abstract

Generative adversarial networks for handwriting image generation: a review

Elanwar,

Betke

2024

Vis Comput

View full text Add to dashboard Cite

Handwriting synthesis, the task of automatically generating realistic images of handwritten text, has gained increasing attention in recent years, both as a challenge in itself, as well as a task that supports handwriting recognition research. The latter task is to synthesize large image datasets that can then be used to train deep learning models to recognize handwritten text without the need for human-provided annotations. While early attempts at developing handwriting generators yielded limited results [1], more recent works involving generative models of deep neural network architectures have been shown able to produce realistic imitations of human handwriting [2–19]. In this review, we focus on one of the most prevalent and successful architectures in the field of handwriting synthesis, the generative adversarial network (GAN). We describe the capabilities, architecture specifics, and performance of the GAN-based models that have been introduced to the literature since 2019 [2–14]. These models can generate random handwriting styles, imitate reference styles, and produce realistic images of arbitrary text that was not in the training lexicon. The generated images have been shown to contribute to improving handwriting recognition results when augmenting the training samples of recognition models with synthetic images. The synthetic images were often hard to expose as non-real, even by human examiners, but also could be implausible or style-limited. The review includes a discussion of the characteristics of the GAN architecture in comparison with other paradigms in the image-generation domain and highlights the remaining challenges for handwriting synthesis.

show abstract

Multimodal robot-assisted English writing guidance and error correction with reinforcement learning

Wang

2024

Front. Neurorobot.

View full text Add to dashboard Cite

IntroductionWith the development of globalization and the increasing importance of English in international communication, effectively improving English writing skills has become a key focus in language learning. Traditional methods for English writing guidance and error correction have predominantly relied on rule-based approaches or statistical models, such as conventional language models and basic machine learning algorithms. While these methods can aid learners in improving their writing quality to some extent, they often suffer from limitations such as inflexibility, insufficient contextual understanding, and an inability to handle multimodal information. These shortcomings restrict their effectiveness in more complex linguistic environments.MethodsTo address these challenges, this study introduces ETG-ALtrans, a multimodal robot-assisted English writing guidance and error correction technology based on an improved ALBEF model and VGG19 architecture, enhanced by reinforcement learning. The approach leverages VGG19 to extract visual features and integrates them with the ALBEF model, achieving precise alignment and fusion of images and text. This enhances the model's ability to comprehend context. Furthermore, by incorporating reinforcement learning, the model can adaptively refine its correction strategies, thereby optimizing the effectiveness of writing guidance.Results and discussionExperimental results demonstrate that the proposed ETG-ALtrans method significantly improves the accuracy of English writing error correction and the intelligence level of writing guidance in multimodal data scenarios. Compared to traditional methods, this approach not only enhances the precision of writing suggestions but also better caters to the personalized needs of learners, thereby effectively improving their writing skills. This research is of significant importance in the field of language learning technology and offers new perspectives and methodologies for the development of future English writing assistance tools.

show abstract

Crosslingual Handwritten Text Generation Using GANs

Cited by 3 publications

References 25 publications

Data Augmentation for Offline Handwritten Text Recognition: A Systematic Literature Review

Data Augmentation for Offline Handwritten Text Recognition: A Systematic Literature Review

Generative adversarial networks for handwriting image generation: a review

Multimodal robot-assisted English writing guidance and error correction with reinforcement learning

Contact Info

Product

Resources

About