Information Extraction from Invoices: A Graph Neural Network Approach for Datasets with High Layout Variety

Krieger, Felix; Drews, Paul; Funk, Burkhardt; Wobbe, Till

doi:10.1007/978-3-030-86797-3_1

Cited by 13 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, graph-based models have been proposed either using Graph Convolutional Networks [33,34] or Graph Attention Networks [35,36] for sequence labelling or node classification to identify the key items. The graph-based document representations include positional relations between text segments but they are limited to the direct neighbours only.…”

Section: Related Workmentioning

confidence: 99%

Information Extraction from Scanned Invoice Images using Text Analysis and Layout Features

Ha,

Horák

2022

Preprint

View full text Add to dashboard Cite

While storing invoice content as metadata to avoid paper document processing may be the future trend, almost all of daily issued invoices are still printed on paper or generated in digital formats such as PDFs. In this paper, we introduce the OCRMiner system for information extraction from scanned document images which is based on text analysis techniques in combination with layout features to extract indexing metadata of (semi-)structured documents. The system is designed to process the document in a similar way a human reader uses, i.e. to employ different layout and text attributes in a coordinated decision. The system consists of a set of interconnected modules that start with (possibly erroneous) character-based output from a standard OCR system and allow to apply different techniques and to expand the extracted knowledge at each step. Using an open source OCR, the system is able to recover the invoice data in 90% for English and in 88% for the Czech set.

show abstract

Section: Related Workmentioning

confidence: 99%

Information Extraction from Scanned Invoice Images using Text Analysis and Layout Features

Ha,

Horák

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…With the advancement of digital transformation, the push for digitizing paper-based documents and reports is ever-increasing, especially in sectors such as healthcare, insurance, and finance [1][2][3]. In this context, the medical examination report (MER) stands out.…”

Section: Introductionmentioning

confidence: 99%

MSIE-Net: Associative Entity-Based Multi-Stage Network for Structured Information Extraction from Reports

Li,

Sheng,

Sheng

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Efficient document recognition and sharing remain challenges in the healthcare, insurance, and finance sectors. One solution to this problem has been the use of deep learning techniques to automatically extract structured information from paper documents. Specifically, the structured extraction of a medical examination report (MER) can enhance medical efficiency, data analysis, and scientific research. While current methods focus on reconstructing table bodies, they often overlook table headers, leading to incomplete information extraction. This paper proposes MSIE-Net (multi-stage-structured information extraction network), a novel structured information extraction method, leveraging refined attention transformers and associated entity detection to enhance comprehensive MER information retrieval. MSIE-Net includes three stages. First, the RVI-LayoutXLM (refined visual-feature independent LayoutXLM) targets key information extraction. In this stage, the refined attention accentuates the interaction between different modalities by adjusting the attention score at the current position using previous position information. This design enables the RVI-LayoutXLM to learn more specific contextual information to improve extraction performance. Next, the associated entity detection module, RIFD-Net (relevant intra-layer fine-tuned detection network), identifies each test item’s location within the MER table body. Significantly, the backbone of RIFD-Net incorporates the intra-layer feature adjustment module (IFAM) to extract global features while homing in on local areas, proving especially sensitive for inspection tasks with dense and long bins. Finally, structured post-processing based on coordinate aggregation links the outputs from the prior stages. For the evaluation, we constructed the Chinese medical examination report dataset (CMERD), based on real medical scenarios. MSIE-Net demonstrated competitive performance in tasks involving key information extraction and associated entity detection. Experimental results validate MSIE-Net’s capability to successfully detect key entities in MER and table images with various complex layouts, perform entity relation extraction, and generate structured labels, laying the groundwork for intelligent medical documentation.

show abstract

“…Information extraction approaches must handle varying layouts, semantic fields and multiple input modalities at the intersection of computer vision, natural language processing and information retrieval. While there has been progress on the task [4,7,14,15,18,19,25,34], there is no publicly available large-scale benchmark to train and compare these approaches, an issue that has been noted by several authors [5,16,24,26,29]. Existing approaches are trained on privately collected datasets, hindering their reproducibility, fair comparisons and tracking field progression [11,23,24].…”

Section: Introductionmentioning

confidence: 99%

DocILE 2023 Teaser: Document Information Localization and Extraction

Šimsa¹,

Müller²,

Skalický³

et al. 2023

Preprint

View full text Add to dashboard Cite

The lack of data for information extraction (IE) from semistructured business documents is a real problem for the IE community. Publications relying on large-scale datasets use only proprietary, unpublished data due to the sensitive nature of such documents. Publicly available datasets are mostly small and domain-specific. The absence of a large-scale public dataset or benchmark hinders the reproducibility and cross-evaluation of published methods. The DocILE 2023 competition, hosted as a lab at the CLEF 2023 conference and as an ICDAR 2023 competition, will run the first major benchmark for the tasks of Key Information Localization and Extraction (KILE) and Line Item Recognition (LIR) from business documents. With thousands of annotated real documents from open sources, a hundred thousand of generated synthetic documents, and nearly a million unlabeled documents, the DocILE lab comes with the largest publicly available dataset for KILE and LIR. We are looking forward to contributions from the Computer Vision, Natural Language Processing, Information Retrieval, and other communities. The data, baselines, code and up-to-date information about the lab and competition are available at https://docile.rossum.ai/.

show abstract

Information Extraction from Invoices: A Graph Neural Network Approach for Datasets with High Layout Variety

Cited by 13 publications

References 10 publications

Information Extraction from Scanned Invoice Images using Text Analysis and Layout Features

Information Extraction from Scanned Invoice Images using Text Analysis and Layout Features

MSIE-Net: Associative Entity-Based Multi-Stage Network for Structured Information Extraction from Reports

DocILE 2023 Teaser: Document Information Localization and Extraction

Contact Info

Product

Resources

About