A synthetic document image dataset for developing and evaluating historical document processing methods

Abstract

Document images accompanied by OCR output text and ground truth transcriptions are useful for developing and evaluating document recognition and processing methods, especially for historical document images. Additionally, research into improving the performance of such methods often requires further annotation of training and test data (e.g., topical document labels). However, transcribing and labeling historical documents is expensive. As a result, existing real-world document image datasets with such accompanying resources are rare and often relatively small. We introduce synthetic document image datasets of varying levels of noise that have been created from standard (English) text corpora using an existing document degradation model applied in a novel way. Included in the datasets is the OCR output from real OCR engines including the commercial ABBYY FineReader and the open-source Tesseract engines. These synthetic datasets are designed to exhibit some of the characteristics of an example real-world document image dataset, the Eisenhower Communiqués. The new datasets also benefit from additional metadata that exist due to the nature of their collection and prior labeling efforts. We demonstrate the usefulness of the synthetic datasets by training an existing multi-engine OCR correction method on the synthetic data and then applying the model to reduce word error rates on the historical document dataset. The synthetic datasets will be made available for use by other researchers.

The NoisyOffice Database: A Corpus To Train Supervised Machine Learning Filters For Image Processing

“…Many different degradation effects can be used as defocusing, paper positioning variations, distortion of character strokes, non-uniform illumination, typesetting imperfections, perspective distortion, etc. [22,20,6,25,21,23,26]. These degradation models aim at generating synthetic noise that can be found in the real world and therefore to extend training sets to perform better on unseen scenarios.…”

How well does multiple OCR error correction generalize?

“…Four datasets were used in this work: two test sets, the Eisenhower Communiqués 30 and the Nineteenth Century Mormon Article Newspaper Index; 31 and two training sets, an extraction of the 2001 Topic Annotated Enron Email Data Set and an extraction of the Reuters-21578 Text Categorization Test Collection. 32,33 The following sections describe each dataset and how it was created.…”

“…For further details on the process, please consult the paper by Walker, Lund, and Ringger (2013). 33…”