Assessing the Impact of OCR Errors in Information Retrieval

Bazzo, Guilherme Torresan; Lorentz, Gustavo Acauan; Vargas, Danny Suarez; Moreira, Viviane Pereira

doi:10.1007/978-3-030-45442-5_13

Cited by 16 publications

(9 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Firstly, experienced OCR quality was the main target of the study instead of studying effects of data-oriented OCR quality, which has been repeated probably tens of times in different collections and languages (e.g. Järvelin et al ., 2016; Bazzo et al ., 2020). As Kumpulainen and Late (2022) show empirically, noise in OCR quality disturbs researchers of historical newspaper collections both in the searching and selection phases of their research.…”

Section: Discussionmentioning

confidence: 99%

“…Simulated research settings include, e.g. Taghva et al (1996), Savoy and Naji (2011) and Bazzo et al (2020), just to mention a few. The general result of these studies is that worse OCR quality lowers query results clearly.…”

Section: Related Researchmentioning

confidence: 99%

“…Even if the improvement in OCR quality is considerable, the improved quality can still be challenging for information retrieval engines, especially with short queries and articles, where the information retrieval engine has less evidence for matching the query words and collection data in the engine's index (J€ arvelin et al, 2016;Mittendorf and Sch€ auble, 2000). In a recent study, Bazzo et al (2020), for example, found that statistically significant degradation of search results begins already at the word error rate of 5%, when Portuguese pdf texts with artificially induced errors were sought for with state-of-the-art modern query engine and algorithms. The same kind of results has been achieved in most of the current studies, as a current survey paper of Nguyen et al (2021) shows.…”

Section: Topic Creation For the Studymentioning

confidence: 99%

See 2 more Smart Citations

Optical character recognition quality affects subjective user perception of historical newspaper clippings

Kettunen

Keskustalo

Kumpulainen

et al. 2023

View full text Add to dashboard Cite

PurposeThis study aims to identify user perception of different qualities of optical character recognition (OCR) in texts. The purpose of this paper is to study the effect of different quality OCR on users' subjective perception through an interactive information retrieval task with a collection of one digitized historical Finnish newspaper.Design/methodology/approachThis study is based on the simulated work task model used in interactive information retrieval. Thirty-two users made searches to an article collection of Finnish newspaper Uusi Suometar 1869–1918 which consists of ca. 1.45 million autosegmented articles. The article search database had two versions of each article with different quality OCR. Each user performed six pre-formulated and six self-formulated short queries and evaluated subjectively the top 10 results using a graded relevance scale of 0–3. Users were not informed about the OCR quality differences of the otherwise identical articles.FindingsThe main result of the study is that improved OCR quality affects subjective user perception of historical newspaper articles positively: higher relevance scores are given to better-quality texts.Originality/valueTo the best of the authors’ knowledge, this simulated interactive work task experiment is the first one showing empirically that users' subjective relevance assessments are affected by a change in the quality of an optically read text.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Researchmentioning

confidence: 99%

Section: Topic Creation For the Studymentioning

confidence: 99%

See 1 more Smart Citation

Optical character recognition quality affects subjective user perception of historical newspaper clippings

Kettunen

Keskustalo

Kumpulainen

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Owing to large variations between collections of handwritten/historical documents, recognition models typically need to be re-trained/fine-tuned on annotated samples from the new set of documents that need to be recognized. A recent study by Bazzo et al [21] observes that even a 5% word error has a significant impact on information retrieval from document images that are automatically transcribed using an OCR. In another study based on data in a large digital library, Chiron et al [22] observe that a significant number of user queries are affected by OCR errors.…”

Section: Introductionmentioning

confidence: 99%

Asking questions on handwritten document collections

Mathew

Gómez

Karatzas

et al. 2021

IJDAR

View full text Add to dashboard Cite

This work addresses the problem of Question Answering (QA) on handwritten document collections. Unlike typical QA and Visual Question Answering (VQA) formulations where the answer is a short text, we aim to locate a document snippet where the answer lies. The proposed approach works without recognizing the text in the documents. We argue that the recognitionfree approach is suitable for handwritten documents and historical collections where robust text recognition is often difficult. At the same time, for human users, document image snippets containing answers act as a valid alternative to textual answers. The proposed approach uses an off-the-shelf deep embedding network which can project both textual words and word images into a common sub-space. This embedding bridges the textual and visual domains and helps us retrieve document snippets that potentially answer a question. We evaluate results of the proposed approach on two new datasets: (i) HW-SQuAD: a synthetic, handwritten document image counterpart of SQuAD1.0 dataset and (ii) BenthamQA: a smaller set of QA pairs defined on documents from the popular Bentham manuscripts collection. We also present a thorough analysis of the proposed recognition-free approach compared to a recognition-based approach which uses text recognized from the images using an OCR. Datasets presented in this work are available to download at docvqa.org.

show abstract

“…Owing to large variations between collections of handwritten/historical documents, recognition models typically need to be retrained/fine-tuned on annotated samples from the new set of documents that need to be recognized. A recent study by Bazzo et al [4] observe that even a 5% word error has a significant impact on information retrieval from document images that are automatically transcribed using an OCR. In another study based on data in a large digital library, Chiron et al [13] observe that a significant number of user queries are affected by OCR errors.…”

Section: Introductionmentioning

confidence: 99%

Asking questions on handwritten document collections

Mathew,

Gomez,

Karatzas

et al. 2021

Preprint

View full text Add to dashboard Cite

This work addresses the problem of Question Answering (QA) on handwritten document collections. Unlike typical QA and Visual Question Answering (VQA) formulations where the answer is a short text, we aim to locate a document snippet where the answer lies. The proposed approach works without recognizing the text in the documents. We argue that the recognition-free approach is suitable for handwritten documents and historical collections where robust text recognition is often difficult. At the same time, for human users, document image snippets containing answers act as a valid alternative to textual answers. The proposed approach uses an off-the-shelf deep embedding network which can project both textual words and word images into a common sub-space. This embedding bridges the textual and visual domains and help us retrieve document snippets that potentially answer a question. We evaluate results of the proposed approach on two new datasets: (i) HW-SQuAD: a synthetic, handwritten document image counterpart of SQuAD1.0 dataset, and (ii) BenthamQA: a smaller set of QA pairs defined on documents from the popular Bentham manuscripts collection. We also present a thorough analysis of the proposed recognition-free approach compared to a recognition-based approach which uses text recognized from the images using an OCR.

show abstract

Assessing the Impact of OCR Errors in Information Retrieval

Cited by 16 publications

References 10 publications

Optical character recognition quality affects subjective user perception of historical newspaper clippings

Optical character recognition quality affects subjective user perception of historical newspaper clippings

Asking questions on handwritten document collections

Asking questions on handwritten document collections

Contact Info

Product

Resources

About