Can Morphological Analyzers Improve the Quality of Optical Character Recognition?

Abstract

Optical Character Recognition (OCR) can substantially improve the usability of digitized documents. Language modeling using word lists is known to improve OCR quality for English. For morphologically rich languages, however, even large word lists do not reach high coverage on unseen text. Morphological analyzers offer a more sophisticated approach, which is useful in many language processing applications. is paper investigates language modeling in the open-source OCR engine Tesseract using morphological analyzers. We present experiments on two Uralic languages Finnish and Erzya. According to our experiments, word lists may still be superior to morphological analyzers in OCR even for languages with rich morphology. Our error analysis indicates that morphological analyzers can cause a large amount of real word OCR errors.

Evaluating the Impact of OCR Errors on Topic Modeling

“…Common OCR errors include punctuation errors, case sensitivity, character format, word meaning and segmentation error where spacings in different line, word or character lead to mis-recognitions of white-spaces [22]. OCR errors may also stem from other sources such as font variation across different materials, historical spelling variations, material quality or language specific to different media texts [1].…”

“…However, historical documents still pose a challenge for character recognition and therefore OCR of such documents still does not yield satisfying results. Some of the reasons why historical documents still pose a challenge include font variation across different materials, same words spelled differently, material quality where some documents can have deformations and unavailability of a lexicon of known historical spelling variants [1]. These factors reduce the accuracy of recognition which affects the processing of the documents and, overall, the use of digital libraries.…”

Nykysuomen sanakirjan digitaalinen editio

“…Työ on jatkoa aiemmille julkaisuille yksittäisten kielellisesti poikkeuksellisesti haastavien aineistojen käsittelystä nykyaikaisin menetelmin. Erityisenä alan pioneerina voidaan pitää Jack Rueteria, jonka aiemmat tutkimukset ovat jo varhain tuoneet uutta tietoa erityisesti morfologisten analysaattoreiden käytöstä tekstintunnistuksessa [28], minkä lisäksi hän on julkaissut ja alkanut avata kohta vuosikymmenen tällaiselle työlle keskeistä aineistoa eri kielillä (katso mm. [24,25,26]).…”

Open-Source Morphology for Endangered Mordvinic Languages

“…After the release first finite-state transducer for the closely related Komi-Zyrian (Rueter, 2000), it was only obvious that similar work should be done for Erzya Mordvin. Fortunately, over the past decade there has been an increasing number of publications on Erzya, relating to its morphology (Rueter, 2010), its OCR tools (Silfverberg and Rueter, 2015) and universal dependencies (Rueter and Tyers, 2018).…”