Assessing and Minimizing the Impact of OCR Quality on Named Entity Recognition

Hamdi, Ahmed; Jean-Caurant, Axel; Sidère, Nicolas; Coustaty, Mickaël; Doucet, Antoine

doi:10.1007/978-3-030-54956-5_7

Cited by 23 publications

(17 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The training on synOCR'd WikiNER/CoNLL gives slightly worse results than NN base too. The corruption of the training data without the usage of any embeddings seems to harm performance drastically, what is in line with the observation of Hamdi et al [13]. It is striking that the training on corrupted/synOCR'd Dutch gives especially bad results for PER compared to French.…”

Section: Cross-domain Setupsupporting

confidence: 83%

“…We test on a subset of the Dutch Europeana corpus. Hamdi et al [13] show that neural taggers perform better compared to other taggers like the Stanford NER tagger and they also prove that performance decreases drastically if the OCR error rate increases. Piktus et al [19] learn misspelling-oblivious FastText embeddings from synthetic misspellings generated by an error model for partof-speech tagging.…”

Section: Related Workmentioning

confidence: 99%

“…They find that the LOC class entities causes the most difficulties in the recognition of named entities. The recent work of Hamdi et al [13] investigates the impact of OCR errors on NER. To do so, they also perturb modern corpora synthetically with different degrees of error rates.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Data Centric Domain Adaptation for Historical Text with OCR Errors

März¹,

Schweter²,

Poerner³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose new methods for in-domain and cross-domain Named Entity Recognition (NER) on historical data for Dutch and French. For the cross-domain case, we address domain shift by integrating unsupervised in-domain data via contextualized string embeddings; and OCR errors by injecting synthetic OCR errors into the source domain and address data centric domain adaptation. We propose a general approach to imitate OCR errors in arbitrary input data. Our cross-domain as well as our in-domain results outperform several strong baselines and establish state-of-the-art results. We publish preprocessed versions of the French and Dutch Europeana NER corpora.

show abstract

Section: Cross-domain Setupsupporting

confidence: 83%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Data Centric Domain Adaptation for Historical Text with OCR Errors

März¹,

Schweter²,

Poerner³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…If sentence segmentation and dependency parsing bear the brunt of low OCR quality, NER is also affected with a significant drop of F-score between good and poor OCR (from 87% to 63% for person entities). Focusing specifically on entity processing, Hamdi et al [79,80] confronted a BiLSTM-based NER model with OCR outputs of the same text but of different qualities and observed a 30 percentage point loss in F-score when the character error rate increased from 7% to 20%. Finally, in order to assess the impact of noisy entities on NER during the CLEF-HIPE-2020 NE evaluation campaign on historical newspapers (HIPE-2020 for short), 7 Ehrmann et al [53] evaluated systems' performances on various entity noise levels, defined as the length-normalised Levenshtein distance between the OCR surface form of an entity and its manual transcription.…”

Section: Character Recognitionmentioning

confidence: 99%

Named Entity Recognition and Classification on Historical Documents: A Survey

Ehrmann¹,

Hamdi²,

Pontes³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

After decades of massive digitisation, an unprecedented amount of historical documents is available in digital format, along with their machine-readable texts. While this represents a major step forward with respect to preservation and accessibility, it also opens up new opportunities in terms of content mining and the next fundamental challenge is to develop appropriate technologies to efficiently search, retrieve and explore information from this 'big data of the past'. Among semantic indexing opportunities, the recognition and classification of named entities are in great demand among humanities scholars. Yet, named entity recognition (NER) systems are heavily challenged with diverse, historical and noisy inputs. In this survey, we present the array of challenges posed by historical documents to NER, inventory existing resources, describe the main approaches deployed so far, and identify key priorities for future developments.

show abstract

“…Still, we can only consider OCR and HTR to be "research facilitators" as long as they perform within reasonable accuracy ranges. Several studies have shown that inaccuracies in OCRed documents harm information retrieval and text mining techniques like named entity recognition and linking, topic modelling, and language modelling (Alex and Burns, 2014;Chiron et al, 2017;van Strien et al, 2020;Hamdi et al, 2020;Pontes et al, 2019;Hill and Hengchen, 2019). But what do we mean by "reasonable accuracy ranges"?…”

Section: Introductionmentioning

confidence: 99%

Evaluation of HTR models without Ground Truth Material

Ströbel¹,

Clematide²,

Völk³

et al. 2022

Preprint

View full text Add to dashboard Cite

The evaluation of Handwritten Text Recognition (HTR) models during their development is straightforward: because HTR is a supervised problem, the usual data split into training, validation, and test data sets allows the evaluation of models in terms of accuracy or error rates. However, the evaluation process becomes tricky as soon as we switch from development to application. A compilation of a new (and forcibly smaller) ground truth (GT) from a sample of the data that we want to apply the model on and the subsequent evaluation of models thereon only provides hints about the quality of the recognised text, as do confidence scores (if available) the models return. Moreover, if we have several models at hand, we face a model selection problem since we want to obtain the best possible result during the application phase. This calls for GT-free metrics to select the best model, which is why we (re-)introduce and compare different metrics, from simple, lexicon-based to more elaborate ones using standard language models and masked language models (MLM). We show that MLM-based evaluation can compete with lexicon-based methods, with the advantage that large and multilingual transformers are readily available, thus making compiling lexical resources for other metrics superfluous.

show abstract

Assessing and Minimizing the Impact of OCR Quality on Named Entity Recognition

Cited by 23 publications

References 27 publications

Data Centric Domain Adaptation for Historical Text with OCR Errors

Data Centric Domain Adaptation for Historical Text with OCR Errors

Named Entity Recognition and Classification on Historical Documents: A Survey

Evaluation of HTR models without Ground Truth Material

Contact Info

Product

Resources

About