Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

Abstract

The unstructured data impacts 95% of the organizations and costs them millions of dollars annually. If managed well, it can significantly improve business productivity. The traditional information extraction techniques are limited in their functionality, but AI-based techniques can provide a better solution. A thorough investigation of AI-based techniques for automatic information extraction from unstructured documents is missing in the literature. The purpose of this Systematic Literature Review (SLR) is to recognize, and analyze research on the techniques used for automatic information extraction from unstructured documents and to provide directions for future research. The SLR guidelines proposed by Kitchenham and Charters were adhered to conduct a literature search on various databases between 2010 and 2020. We found that: 1. The existing information extraction techniques are template-based or rule-based, 2. The existing methods lack the capability to tackle complex document layouts in real-time situations such as invoices and purchase orders, 3.The datasets available publicly are task-specific and of low quality. Hence, there is a need to develop a new dataset that reflects real-world problems. Our SLR discovered that AI-based approaches have a strong potential to extract useful information from unstructured documents automatically. However, they face certain challenges in processing multiple layouts of the unstructured documents. Our SLR brings out conceptualization of a framework for construction of high-quality unstructured documents dataset with strong data validation techniques for automated information extraction. Our SLR also reveals a need for a close association between the businesses and researchers to handle various challenges of the unstructured data analysis.

Document image analysis and recognition: a survey

“…But older publications do not account for the technological shift of recent 10 years, when the full DIA process from image or video capture to full recognition and results presentation became possible directly on mobile or autonomous devices. At the same time, to the best of our knowledge, publications of recent years considered only separate tasks (such as document image classification [3], extraction of information from poorly structured documents [4], etc.) or advances of particular methods (mostly machine [16] and deep [20,21]…”

Multi-Layout Unstructured Invoice Documents Dataset: A Dataset for Template-Free Invoice Processing and Its Evaluation Using AI Approaches

“…The automatic and efficient key field extraction task is one of the challenging tasks as its solution is spanned across the use of Computer Vision (CV) and Natural Language Processing (NLP) [6]. The unstructured documents such as invoices, claim processing forms usually do not comprise "natural language" as other regular documents or paragraphs.…”

“…Few of the challenges mentioned above, can be solved using Deep Learning (DL) approaches [6]. Automatic feature extraction and availability of pre-trained Neural Networks (NN) trained on huge unlabeled corpus are the main advantages of using DL approaches in information extraction tasks.…”

Application of NLP-based topic modeling to analyse unstructured text data in annual reports of construction contracting companies

“…Specifically, in the case of publicly listed private firms, annual reports and financial statements are mandatory disclosures in the public domain. Knowledge extraction from such unstructured data is now possible with the recent developments in computer-aided text mining and Natural Language Processing (NLP) [ 6 – 8 ]. In this research, the authors explore the efficiency of NLP-based topic modeling algorithms to extract keywords and topics from the publicly available annual reports of construction contracting firms and use the information obtained to analyze the strategies such firms adopt in dealing with emerging sectoral challenges explained in the next section.…”