Visual Detection with Context for Document Layout Analysis

Soto, Carlos; Yoo, Shinjae

doi:10.18653/v1/d19-1348

Cited by 55 publications

(39 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 3 provides a comparison of the DocBank to the previous document layout analysis datasets, including Article Regions (Soto and Yoo, 2019), GROTOAP2 (Tkaczyk et al, 2014), PubLayNet (Zhong et al, 2019), and TableBank (Li et al, 2019 19.36% Total 400,000 100.00% 50,000 100.00% 50,000 100.00% 500,000 100.00%…”

Section: Dataset Statisticsmentioning

confidence: 99%

DocBank: A Benchmark Dataset for Document Layout Analysis

Li¹,

Xu²,

Cui³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

144

View full text Add to dashboard Cite

Document layout analysis usually relies on computer vision models to understand documents while ignoring textual information that is vital to capture. Meanwhile, high quality labeled datasets with both visual and textual information are still insufficient. In this paper, we present DocBank, a benchmark dataset that contains 500K document pages with fine-grained tokenlevel annotations for document layout analysis. DocBank is constructed using a simple yet effective way with weak supervision from the L A T E X documents available on the arXiv.com. With DocBank, models from different modalities can be compared fairly and multi-modal approaches will be further investigated and boost the performance of document layout analysis. We build several strong baselines and manually split train/dev/test sets for evaluation. Experiment results show that models trained on DocBank accurately recognize the layout information for a variety of documents. The DocBank dataset is publicly available at https: //github.com/doc-analysis/DocBank.

show abstract

Section: Dataset Statisticsmentioning

confidence: 99%

DocBank: A Benchmark Dataset for Document Layout Analysis

Li¹,

Xu²,

Cui³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

144

View full text Add to dashboard Cite

show abstract

“…Nowadays, deep learning methods have become the mainstream for many machine learning problems (Yang et al, 2017;Borges Oliveira and Viana, 2017;Katti et al, 2018;Soto and Yoo, 2019). (Yang et al, 2017) propose a pixel-by-pixel classification to solve the document semantic structure extraction problem.…”

Section: Deep Learning Approachesmentioning

confidence: 99%

“…In this way, the model significantly outperforms approaches based on sequential text or document images. In addition, (Soto and Yoo, 2019) incorporate contextual information into the Faster R-CNN model. They involve the inherently localized nature of article contents to improve region detection performance.…”

Section: Deep Learning Approachesmentioning

confidence: 99%

StructuralLM: Structural Pre-training for Form Understanding

Li¹,

Bi²,

Yan³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, they almost exclusively focus on text-only representation, while neglecting cell-level layout information that is important for form image understanding. In this paper, we propose a new pre-training approach, StructuralLM, to jointly leverage cell and layout information from scanned documents. Specifically, we pre-train StructuralLM with two new designs to make the most of the interactions of cell and layout information: 1) each cell as a semantic unit; 2) classification of cell positions. The pre-trained StructuralLM achieves new state-of-the-art results in different types of downstream tasks, including form understanding (from 78.95 to 85.14), document visual question answering (from 72.59 to 83.94) and document image classification (from 94.43 to 96.08).

show abstract

“…Early methods apply markov random fields [7] or conditional random fields [2] to solve the problem. More recent approaches tend to use CNN [14,22,30], GNN [10], and Bert [28] to improve performance. Despite their good performance, these methods require large amounts of training data which is hard to collect due to privacy reasons.…”

Section: Text Field Labelingmentioning

confidence: 99%

“…The task aims to assign a label to each text region in a document so that text information could be extracted in structured formats. Learning based methods [2,10,14,22,30] are shown to have good performance for text field labeling. They could automatically adapt to any type of layouts, but they usually require sufficient training data.…”

Section: Introductionmentioning

confidence: 99%

One-shot Text Field labeling using Attention and Belief Propagation for Structure Information Extraction

Cheng

Qiu

Shi

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Structured information extraction from document images usually consists of three steps: text detection, text recognition, and text field labeling. While text detection and text recognition have been heavily studied and improved a lot in literature, text field labeling is less explored and still faces many challenges. Existing learning based methods for text labeling task usually require a large amount of labeled examples to train a specific model for each type of document. However, collecting large amounts of document images and labeling them is difficult and sometimes impossible due to privacy issues. Deploying separate models for each type of document also consumes a lot of resources. Facing these challenges, we explore one-shot learning for the text field labeling task. Existing one-shot learning methods for the task are mostly rule-based and have difficulty in labeling fields in crowded regions with few landmarks and fields consisting of multiple separate text regions. To alleviate these problems, we proposed a novel deep end-to-end trainable approach for one-shot text field labeling, which makes use of attention mechanism to transfer the layout information between document images. We further applied conditional random field on the transferred layout information for the refinement of field labeling. We collected and annotated a real-world one-shot field labeling dataset with a large variety of document types and conducted extensive experiments to examine the effectiveness of the proposed model. To stimulate research in this direction, the collected dataset and the one-shot model will be released 1. CCS CONCEPTS • Applied computing → Document analysis; Optical character recognition; • Computing methodologies → Visual contentbased indexing and retrieval.

show abstract

Visual Detection with Context for Document Layout Analysis

Cited by 55 publications

References 20 publications

DocBank: A Benchmark Dataset for Document Layout Analysis

DocBank: A Benchmark Dataset for Document Layout Analysis

StructuralLM: Structural Pre-training for Form Understanding

One-shot Text Field labeling using Attention and Belief Propagation for Structure Information Extraction

Contact Info

Product

Resources

About