Manifold Mixup Improves Text Recognition with CTC Loss

Abstract

Modern handwritten text recognition techniques employ deep recurrent neural networks. The use of these techniques is especially efficient when a large amount of annotated data is available for parameter estimation. Data augmentation can be used to enhance the performance of the systems when data is scarce. Manifold Mixup is a modern method of data augmentation that meld two images or the feature maps corresponding to these images and the targets are fused accordingly. We propose to apply the Manifold Mixup to text recognition while adapting it to work with a Connectionist Temporal Classification cost. We show that Manifold Mixup improves text recognition results on various languages and datasets.

Data Augmentation for Offline Handwritten Text Recognition: A Systematic Literature Review

“…The historical Bentham dataset [61], which consists of images of letters by the English philosopher Jeremy Bentham (1748-1832), was utilized in the work of [47]. Furthermore, the English subset of the Maurdor dataset [62] was used in [34] and contains heterogeneous images of different types of documents. Finally, the dataset "GoodNotes Handwriting Kollection" (GNHK) [63] comprises unrestricted cameracaptured images of English handwritten text from various regions, characterized by diverse styles and increased noise, was used in the work of [38].…”

“…The RIMES dataset was used in the studies [27, 29-31, 34, 37, 40, 42, 46, 49]. The second dataset was the French subset of Maurdor, which was utilized in the study [34].…”

“…The Arabic language was also significantly utilized, offering substantial variation across datasets. A first remarkable dataset in Arabic is the Maurdor [62] subset, which has approximately 13,000 text-line samples and was used in the study of [34]. OpenHaRT [66], boasting a large database of approximately 710,000 images, was utilized in the study of [46].…”

“…The "Handwritten Kazakh and Russian" (HKR) dataset [72], representing Kazakh and Russian languages, has been utilized in the studies by [35,41,43]. Furthermore, the Chinese language is also represented by the dataset from the "Chinese Academy of Sciences' Institute of Automation" (CASIA) [73], that was used in the works of [34,37,50,53]. CASIA offers online and offline recognition versions.…”

Music-Guided Video Summarization using Quadratic Assignments

“…Other point cloud Mixup methods include Rigid SubSet Mixup [29] and Point MixSwap [49]. Mixup has also been investigated for LiDAR [55], graphs [53], speaker verification [70], vision-language navigation [32], single-view 3D reconstruction [10], and language processing [28,38,45,54,68]. We focus on Mixup for images, but our approach is generic and can be applied to many Mixup variants.…”