Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout

Graliński, Filip; Stanisławek, Tomasz; Wróblewska, Anna; Lipiński, Dawid; Kaliska, Agnieszka; Rosalska, Paulina; Topolski, Bartosz; Biecek, Przemysław

doi:10.48550/arxiv.2003.02356

Cited by 7 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DocFormer achieves 96.33% F1 on this dataset besting all prior *-base and virtually all *large variants tying with TILT-large [40] which has higher number of parameters. Kleister-NDA [16]: dataset consists of legal NDA documents. The task with Kleister-NDA data is to extract the values of four fixed labels.…”

Section: Entity Extraction Taskmentioning

confidence: 99%

“…We present all the hyper-parameters in Table 11 used for pre-training and fine-tuning DocFormer . We fine-tune on downstream tasks on the same number of epochs as prior art [55,56,25]: FUNSD [17], Kleister-NDA [16] datasets were fine-tuned for 100 epochs. CORD [46] for 200 epochs.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…In this sub-section we look at CORD [46] visualizations by DocFormer . We expllicitly show hard-cases where Doc-Former does well, see Figures 12,13,14,15,16. In order to be transparent, we also show an error scenario in Figure 17. Legend for the colors in images is, Menu items: Red, Total: Blue, Sub-total (pre-tax): Green, Void-menu: Cyan color, Other: grey.…”

Section: Cord Visualizationsmentioning

confidence: 99%

See 2 more Smart Citations

DocFormer: End-to-End Transformer for Document Understanding

Appalaraju

Jasani

Kota

et al. 2021

Preprint

View full text Add to dashboard Cite

We present DocFormer -a multi-modal transformer based architecture for the task of Visual Document Understanding (VDU). VDU is a challenging problem which aims to understand documents in their varied formats (forms, receipts etc.) and layouts. In addition, DocFormer is pre-trained in an unsupervised fashion using carefully designed tasks which encourage multi-modal interaction. DocFormer uses text, vision and spatial features and combines them using a novel multi-modal self-attention layer. DocFormer also shares learned spatial embeddings across modalities which makes it easy for the model to correlate text to visual tokens and vice versa. DocFormer is evaluated on 4 different datasets each with strong baselines. DocFormer achieves state-of-the-art results on all of them, sometimes beating models 4x its size (in no. of parameters).

show abstract

Section: Entity Extraction Taskmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

Section: Cord Visualizationsmentioning

confidence: 99%

See 1 more Smart Citation

DocFormer: End-to-End Transformer for Document Understanding

Appalaraju

Jasani

Kota

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The standard approach to information extraction is a two stage process, that requires an initial OCR step followed by a second information localization step. Localization can be performed on the extracted text sequence, by training a NER model [5]. With such an approach however spatial information is lost.…”

Section: Related Workmentioning

confidence: 99%

DocReader: Bounding-Box Free Training of a Document Information Extraction Model

Klaiman¹,

Lehne²

2021

Preprint

View full text Add to dashboard Cite

Information extraction from documents is a ubiquitous first step in many business applications. During this step, the entries of various fields must first be read from the images of scanned documents before being further processed and inserted into the corresponding databases. While many different methods have been developed over the past years in order to automate the above extraction step, they all share the requirement of bounding-box or text segment annotations of their training documents. In this work we present DocReader, an end-to-end neuralnetwork-based information extraction solution which can be trained using solely the images and the target values that need to be read. The DocReader can thus leverage existing historical extraction data, completely eliminating the need for any additional annotations beyond what is naturally available in existing human-operated service centres. We demonstrate that the DocReader can reach and surpass other methods which require bounding-boxes for training, as well as provide a clear path for continual learning during its deployment in production.

show abstract

“…Visually rich document understanding includes many tasks, such as layout recognization (Zhong et al, 2019b;Li et al, 2020), table detection and recognition (Li et al, 2019a;Zhong et al, 2019a) and key information extraction (Graliński et al, 2020;Guo et al, 2019;Huang et al, 2019;G. Jaume and Thiran, 2019;Majumder et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Entity Relation Extraction as Dependency Parsing in Visually Rich Documents

Zhang¹,

Zhang²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Previous works on key information extraction from visually rich documents (VRDs) mainly focus on labeling the text within each bounding box (i.e., semantic entity), while the relations in-between are largely unexplored. In this paper, we adapt the popular dependency parsing model, the biaffine parser, to this entity relation extraction task. Being different from the original dependency parsing model which recognizes dependency relations between words, we identify relations between groups of words with layout information instead. We have compared different representations of the semantic entity, different VRD encoders, and different relation decoders. For the model training, we explore multi-task learning to combine entity labeling and relation extraction tasks; and for the evaluation, we conduct experiments on different datasets with filtering and augmentation. The results demonstrate that our proposed model achieves 65.96% F1 score on the FUNSD dataset. As for the realworld application, our model has been applied to the in-house customs data, achieving reliable performance in the production setting. * Corresponding author. The author's contributions were carried out while at Alibaba Group. His current affiliation is Vipshop (China) Co., Ltd.

show abstract

Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout

Cited by 7 publications

References 8 publications

DocFormer: End-to-End Transformer for Document Understanding

DocFormer: End-to-End Transformer for Document Understanding

DocReader: Bounding-Box Free Training of a Document Information Extraction Model

Entity Relation Extraction as Dependency Parsing in Visually Rich Documents

Contact Info

Product

Resources

About