Statistical learning for OCR error correction

Jatowt

2019 International Conference on Document Analysis and Recognition (ICDAR)

et al. 2019

The accuracy of Optical Character Recognition (OCR) technologies considerably impacts the way digital documents are indexed, accessed and exploited. Post-processing approaches detect and correct remaining errors to improve the quality of OCR texts. However, state-of-the-art approaches still need to be improved. Most of the existing post-OCR techniques use predefined error position lists or apply simple techniques to detect errors. In this paper, we describe a novel error detector using different features from character-level (including character noisy channel, index of peculiarity) to word-level (such as frequencies of n-grams, skip-grams, part-of-speech) Experimental results show that our approach outperforms the best performing techniques in the ICDAR 2017 Competition on Post-OCR text correction.

Section: Mixed Error Detectionmentioning

confidence: 99%

Post-OCR Error Detection by Generating Plausible Candidates

Jatowt

2019 International Conference on Document Analysis and Recognition (ICDAR)

et al. 2019

“…Most of the techniques of this type rely on noisy channel and language model [1,15,27]. The others explore several machine learning techniques to suggest correct candidates [2,10,16].…”

Section: Ocr Post-processing Approachesmentioning

confidence: 99%

“…Other approaches [10,16] explored different sources to generate candidates and then ranked them using a regression model. Several features were extracted such as confusion probability, uni-gram frequency, context feature, term frequency in the OCR text, word confidence, and string similarity.…”

Section: Ocr Post-processing Approachesmentioning

confidence: 99%

“…Some mainly focused on real-word and non-word errors [27]. Other approaches considered errors with segmentation at word or character level [10,16]. Lastly, some others [1,2,7,[24][25][26] just gave examples of OCR errors without any detailed statistics.…”

Section: Figure 1: Error Rates Based On Edit Operation Typesmentioning

confidence: 99%

“…In order to transform token A to token B, four basic edit operation types can be performed: deletion, insertion, substitution, and transposition [6]. Prior works [11,16,27] indicated that transposition is common in misspellings but rarely occurs in OCR errors. We then only consider the three first types.…”

Section: Edit Operationsmentioning

confidence: 99%

See 2 more Smart Citations

Deep Statistical Analysis of OCR Errors for Effective Post-OCR Processing

Jatowt

2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL)

et al. 2019

Post-OCR is an important processing step that follows optical character recognition (OCR) and is meant to improve the quality of OCR documents by detecting and correcting residual errors. This paper describes the results of a statistical analysis of OCR errors on four document collections. Five aspects related to general OCR errors are studied and compared with human-generated misspellings, including edit operations, length effects, erroneous character positions, real-word vs. non-word errors, and word boundaries. Based on the observations from the analysis we give several suggestions related to the design and implementation of effective OCR post-processing approaches.

Adaptive Edit-Distance and Regression Approach for Post-OCR Text Correction

Lecture Notes in Computer Science

Doucet

et al. 2018

Post-processing is a crucial step in improving the performance of OCR process. In this paper, we present a novel approach which explores a modified way of candidate generating and candidate scoring at character level as well as word level. These features are combined with some important features suggested by related work for ranking candidates in a regression model. The experimental results show that our approach has comparable results with the top performing approaches in the Post-OCR text correction competition ICDAR 2017.