A Survey on Various OCR Errors

Kumar, Atul

doi:10.5120/ijca2016910142

Cited by 6 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Repeating the whole pipeline using a paid OCR resource like Abbyy FineReader, which our library already pays to access, could yield more accurate raw text. If further OCR correction seems necessary, there are various NLP and statistical methods that have been researched for correcting OCR errors computationally (Amrhein & Clematide, 2018;Kumar, 2016). Excluding Latin text and sentence fragments is another non-trivial step.…”

Section: Future Improvementsmentioning

confidence: 99%

Using NLP to Generate MARC Summary Fields for Notre Dame ’s Catholic Pamphlets

Flannery

2020

IJoL

View full text Add to dashboard Cite

Three NLP (Natural Language Processing) automated summarization techniques were tested on a special collection of Catholic Pamphlets acquired by Hesburgh Libraries. The automated summaries were generated after feeding the pamphlets as .pdf files into an OCR pipeline. Extensive data cleaning and text preprocessing were necessary before the computer summarization algorithms could be launched. Using the standard ROUGE F1 scoring technique, the Bert Extractive Summarizer technique had the best summarization score. It most closely matched the human reference summaries. The BERT Extractive technique yielded an average Rouge F1 score of 0.239. The Gensim python package implementation of TextRank scored at .151. A hand-implemented TextRank algorithm created summaries that scored at 0.144. This article covers the implementation of automated pipelines to read PDF text, the strengths and weakness of automated summarization techniques, and what the successes and failures of these summaries mean for their potential to be used in Hesburgh Libraries.

show abstract

Section: Future Improvementsmentioning

confidence: 99%

Using NLP to Generate MARC Summary Fields for Notre Dame ’s Catholic Pamphlets

Flannery

2020

IJoL

View full text Add to dashboard Cite

show abstract

“…For example, tokenization errors are known to be frequent in the ACL anthology corpus (Nastase and Hitschler, 2018) and in digitized newspapers (Soni et al, 2019;Adesam et al, 2019). There is large body of research on OCR error correction (Kumar, 2016). However, not all methods can deal with tokenization errors, and it is stated in Hämäläinen and Hengchen (2019) that: "A limitation of our approach is that it cannot do word segmentation in case multiple words have been merged together as a result of the OCR process.…”

Section: Sources Of Tokenization Errorsmentioning

confidence: 99%

Tokenization Repair in the Presence of Spelling Errors

Bast,

Hertel,

Mohamed

2020

Preprint

View full text Add to dashboard Cite

We consider the following tokenization repair problem: Given a natural language text with any combination of missing or spurious spaces, correct these. Spelling errors can be present, but it's not part of the problem to correct them. For example, given: "Tispa per isabout token izaionrep air", compute "Tis paper is about tokenizaion repair".It is tempting to think of this problem as a special case of spelling correction or to treat the two problems together. We make a case that tokenization repair and spelling correction should and can be treated as separate problems. We investigate a variety of neural models as well as a number of strong baselines. We identify three main ingredients to high-quality tokenization repair: deep language models with a bidirectional component, training the models on text with spelling errors, and making use of the space information already present.Our best methods can repair all tokenization errors on 97.5% of the correctly spelled test sentences and on 96.0% of the misspelled test sentences. With all spaces removed from the given text (the scenario from previous work), the accuracy falls to 94.5% and 90.1%, respectively. We conduct a detailed error analysis.

show abstract

“…OCR quality. OCR is a challenge for NLP tasks across research areas [9,10], including for scanned biodiversity texts, which often include handwritten field notes [8,11,12]. Particularly for older documents, the text will likely require additional human effort to clean errors and/or transcribe the documents prior to processing.…”

Section: Preliminary Findingsmentioning

confidence: 99%

Geoparsing biodiversity heritage library collections: A preliminary exploration

Stahlman¹,

Sheffield²

2019

iConference 2019 Proceedings

View full text Add to dashboard Cite

0000−0001−8814−863X] and Carolyn Sheffield 2[0000−0001−9704−7363]Abstract. A short pilot study was conducted to provide recommendations on methods and workflows for extracting geographic references from the text of Biodiversity Heritage Library collections and disambiguating these references. An initial survey of the literature was conducted, and a variety of possible techniques and software were subsequently explored for natural language processing, machine learning, document annotation, and map visualization. A test corpus was evaluated, and preliminary findings identify challenges for a full-scale effort towards automated geoparsing, including: varying OCR quality, diversity of the corpus, historical context, and ambiguity of geographic references. The project background, approaches, and preliminary assessment are described here.

show abstract

A Survey on Various OCR Errors

Cited by 6 publications

References 11 publications

Using NLP to Generate MARC Summary Fields for Notre Dame ’s Catholic Pamphlets

Using NLP to Generate MARC Summary Fields for Notre Dame ’s Catholic Pamphlets

Tokenization Repair in the Presence of Spelling Errors

Geoparsing biodiversity heritage library collections: A preliminary exploration

Contact Info

Product

Resources

About