A unified scheme of text localization and structured data extraction for joint OCR and data mining

Ye, Yibin; Zhu, Shenggao; Wang, Jing; Du, Qi; Yang, Yezhang; Tu, Dandan; Wang, Lanjun; Luo, Jiebo

doi:10.1109/bigdata.2018.8622129

Cited by 9 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Few printed documents like the receipts and the invoices may contain handwritten remarks or characters or numerals, requiring advanced OCR techniques for the recognition [7]. Datasets are also available in different languages like Chinese passport and medical receipt dataset [24]. Language-ambiguity, poor morphology, language-dependent annotations, and the unavailability of large labeled and annotated corpus in the public dataset could be an issue [115], [116].…”

Section: ) Challenges/issues With Existing Datasetsmentioning

confidence: 99%

“…We will cover OCR, RPA and NER as the three approaches/techniques. Text extraction is the main stage in automating document image processing [24], [32], [50], [109]. The document images can be compressed or uncompressed, grayscale or color and the text in the images can be editable or non-editable [64], [59], [39].…”

Section: Rq4-ai Approaches Used For Unstructured Document Processingmentioning

confidence: 99%

“…a character and a classifier with some decision-rules to classify characters [33]. The structural features are less tolerant to character rotation and translation [24]. • Statistical approach: A character is denoted as a numeric feature vector in the statistical approach [49], [90].…”

Section: • •mentioning

confidence: 99%

“…• Neural Networks: Few studies focus on using Neural Networks (NN) to recognize characters [9], [64]. Recurrent Neural Network (RNN) and CNN are used for character classification and recognition for almost all languages [24], [38]. The disadvantage of NN in OCR is their limited prediction and generalization capabilities.…”

Section: • •mentioning

confidence: 99%

“…• OCR helps the organizations to cut down the cost of hiring professionals to carry out data extraction [24]. • Manual data entry is error-prone.…”

Section: Category Of Ocr Enginesmentioning

confidence: 99%

See 4 more Smart Citations

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

et al. 2021

View full text Add to dashboard Cite

The unstructured data impacts 95% of the organizations and costs them millions of dollars annually. If managed well, it can significantly improve business productivity. The traditional information extraction techniques are limited in their functionality, but AI-based techniques can provide a better solution. A thorough investigation of AI-based techniques for automatic information extraction from unstructured documents is missing in the literature. The purpose of this Systematic Literature Review (SLR) is to recognize, and analyze research on the techniques used for automatic information extraction from unstructured documents and to provide directions for future research. The SLR guidelines proposed by Kitchenham and Charters were adhered to conduct a literature search on various databases between 2010 and 2020. We found that: 1. The existing information extraction techniques are template-based or rule-based, 2. The existing methods lack the capability to tackle complex document layouts in real-time situations such as invoices and purchase orders, 3.The datasets available publicly are task-specific and of low quality. Hence, there is a need to develop a new dataset that reflects real-world problems. Our SLR discovered that AI-based approaches have a strong potential to extract useful information from unstructured documents automatically. However, they face certain challenges in processing multiple layouts of the unstructured documents. Our SLR brings out conceptualization of a framework for construction of high-quality unstructured documents dataset with strong data validation techniques for automated information extraction. Our SLR also reveals a need for a close association between the businesses and researchers to handle various challenges of the unstructured data analysis.

show abstract

Section: ) Challenges/issues With Existing Datasetsmentioning

confidence: 99%

Section: Rq4-ai Approaches Used For Unstructured Document Processingmentioning

confidence: 99%

Section: • •mentioning

confidence: 99%

Section: • •mentioning

confidence: 99%

“…• OCR helps the organizations to cut down the cost of hiring professionals to carry out data extraction [24]. • Manual data entry is error-prone.…”

Section: Category Of Ocr Enginesmentioning

confidence: 99%

See 3 more Smart Citations

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

et al. 2021

View full text Add to dashboard Cite

show abstract

Automated text detection from big data scene videos in higher education: a practical approach for MOOCs case study

Devi

Maddala²,

Raju³

2021

J Comput High Educ

View full text Add to dashboard Cite

Multi-Layout Unstructured Invoice Documents Dataset: A Dataset for Template-Free Invoice Processing and Its Evaluation Using AI Approaches

2021

View full text Add to dashboard Cite

The daily transaction of an organization generates a vast amount of unstructured data such as invoices and purchase orders. Managing and analyzing unstructured data is a costly affair for the organization. Unstructured data has a wealth of hidden valuable information. Extracting such insights automatically from unstructured documents can significantly increase the productivity of an organization. Thus, there is a huge demand to develop a tool that can automate the extraction of key fields from unstructured documents. Researchers have used different approaches for extracting key fields, but the lack of annotated and highquality datasets is the biggest challenge. Existing work in this area has used standard and custom datasets for extracting key fields from unstructured documents. Still, the existing datasets face some serious challenges, such as poor-quality images, domain-related datasets, and a lack of data validation approaches to evaluate data quality. This work highlights the detailed process flow for endto-end key fields extraction from unstructured documents. This work presents a high-quality, multi-layout unstructured invoice documents dataset assessed with a statistical data validation technique. The proposed multi-layout unstructured invoice documents dataset is highly diverse in invoice layouts to generalize key field extraction tasks for unstructured documents. The proposed multilayout unstructured invoice documents dataset is evaluated with various feature extraction techniques such as Glove, Word2Vec, FastText, and AI approaches such as BiLSTM and BiLSTM-CRF. We also present the comparative analysis of feature extraction techniques and AI approaches on the proposed multi-layout unstructured invoice document dataset. We attained the best results with BiLSTM-CRF model. INDEX TERMS Artificial Intelligence (AI), information extraction, key field extraction, Named Entity Recognition (NER), template-free invoice processing, unstructured data.

show abstract

A unified scheme of text localization and structured data extraction for joint OCR and data mining

Cited by 9 publications

References 30 publications

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

Automated text detection from big data scene videos in higher education: a practical approach for MOOCs case study

Multi-Layout Unstructured Invoice Documents Dataset: A Dataset for Template-Free Invoice Processing and Its Evaluation Using AI Approaches

Contact Info

Product

Resources

About