A comprehensive evaluation methodology for noisy historical document recognition techniques

Abstract

In this paper, we propose a new comprehensive methodology in order to evaluate the performance of noisy historical document recognition techniques. We aim to evaluate not only the final noisy recognition result but also the main intermediate stages of text line, word and character segmentation. For this purpose, we efficiently create the text line, word and character segmentation ground truth guided by the transcription of the historical documents. The proposed methodology consists of (i) a semiautomatic procedure in order to detect the text line, word and character segmentation ground truth regions making use of the correct document transcription, (ii) calculation of proper evaluation metrics in order to measure the performance of the final OCR result as well as of the intermediate segmentation stages. The semi-automatic procedure for detecting the ground truth regions has been evaluated and proved efficient and time saving. Experimental results prove that using the proposed technique, the percentage of time saved for the text line, word and character segmentation ground truth creation is more than 90%. An analytic experiment using a commercial OCR engine applied to a historical book is also presented.

A Multi-fonts Kanji Character Recognition Method for Early-modern Japanese Printed Books with Ruby Characters

“…In black pixel projection histogram (N. Stamatopoulos and Gatos, 2009), characters are separated by cut-ting at turning points of the cross direction histogram. Figure 4 shows the black pixel projection histogram of a Kanji character.…”

“…As far as we know, any ruby removal methods for earlymodern Japanese printed books have not been studied. As for existing methods to remove ruby characters from current books with standard typography, there are two main methods (N. Stamatopoulos and Gatos, 2009) (Fletcher and Kasturi, 1988): (1) Separating ruby characters linearly using density histogram and (2) separating ruby characters using circumscription rectangles. Both methods assume the standard typography for the target books.…”

Ocropodium: open source OCR for small-scale historical archives

“…Even in relatively recent (e.g. early twentieth century) documents, typography, printing and language can differ widely from modern usage [3].…”

An open-source OCR evaluation tool

“…Other tools are however oriented to evaluate the accuracy in the interpretation of the content (the printed characters and words). This type of evaluation compares the output with a reference which contains a highly accurate transcription (or ground truth) of the source document [12]. The creation of such ground truth is expensive but usually a limited size one is enough to obtain significant numbers provided that it contains a representative sample of the collection.…”