Transfer Learning for OCRopus Model Training on Early Printed Books

Abstract

A method is presented that significantly reduces the character error rates for OCR text obtained from OCRopus models trained on early printed books when only small amounts of diplomatic transcriptions are available. This is achieved by building from already existing models during training instead of starting from scratch. To overcome the discrepancies between the set of characters of the pretrained model and the additional ground truth the OCRopus code is adapted to allow for alphabet expansion or reduction. The character set is now capable of flexibly adding and deleting characters from the pretrained alphabet when an existing model is loaded. For our experiments we use a self-trained mixed model on early Latin prints and the two standard OCRopus models on modern English and German Fraktur texts. The evaluation on seven early printed books showed that training from the Latin mixed model reduces the average amount of errors by 43% and 26%, compared to training from scratch with 60 and 150 lines of ground truth, respectively. Furthermore, it is shown that even building from mixed models trained on standard data unrelated to the newly added training and test data can lead to significantly improved recognition results.

OCR4all—An Open-Source Tool Providing a (Semi-)Automatic OCR Workflow for Historical Printings

“…By dividing the GT in N different folds and aligning them in a certain way, it is possible to train N strong but also diverse models which act as voters in a newly created confidence voting scheme. Second, the so-called pretraining functionality allows to build from an already available Calamari model instead of starting training from scratch which not only speeds up the training process considerably but also improves the recognition accuracy [11]. Third, data augmentation using the routines of ocrodeg 10 for generating noisy variations of training material.…”

“…On the methodical side several improvements have been made by the introduction of voting ensembles, trained with a single OCR engine, whose results are suitably combined [10], and by a pretraining approach which allows to use existing models instead of training from scratch [11].…”

Improving OCR Accuracy on Early Printed Books by combining Pretraining, Voting, and Active Learning

“…Obviously, these examples of transfer learning used far deeper networks than OCRopus with only a single hidden layer, resulting in a dramatically increased number of parameters and consequently more opportunities to learn and remember useful low-level features. Nonetheless, since scripts in general should show a high degree of similarity we still expected a noteworthy impact of pretraining and studied the effect of building from an already available mixed model instead of starting training from scratch (see Reul et al, 2017d). As starting points we used the models for modern English, German Fraktur from the 19th century, and the Latin Antiqua model described above.…”

“…In order to avoid blind spots especially when dealing with small amounts of GT and important but less frequent characters like numbers or capital letters it is possible to define a so-called whitelist. Characters on the whitelist will not get deleted from the matrix no matter if they occur within the GT or not (Reul et al, 2017d).…”

“…Using an already trained model as a starting point for subsequent training with additional individual material requires the capability to add specific characters not previously included in the symbol set (the codec) and the dynamic expansion (and reduction) of the output layer of the neural network. In the context of recurrent neural networks this was recently made possible by Christoph Wick 1 as reported in Reul et al (2017d). Voting is a well known method of classifier combination resulting in less errors than the best single classifier output (Rice et al, 1992).…”