Document Structure Extraction Using Prior Based High Resolution Hierarchical Semantic Segmentation

Sarkar, Mausoom; Aggarwal, Milan; Jain, Arneh; Gupta, Hiresh; Krishnamurthy, Balaji

doi:10.1007/978-3-030-58604-1_39

Cited by 11 publications

(11 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following the setting of the paper [15,64,65,66], we evaluate our model using accuracy, precision, recall and F1 as metrics.…”

Section: Methodsmentioning

confidence: 99%

Human-In-The-Loop Document Layout Analysis

Wu,

Ma,

et al. 2021

Preprint

View full text Add to dashboard Cite

Document layout analysis (DLA) aims to divide a document image into different types of regions. DLA plays an important role in the document content understanding and information extraction systems. Exploring a method that can use less data for effective training contributes to the development of DLA. We consider a Human-in-the-loop (HITL) collaborative intelligence in the DLA. Our approach was inspired by the fact that the HITL push the model to learn from the unknown problems by adding a small amount of data based on knowledge. The HITL select key samples by using confidence. However, using confidence to find key samples is not suitable for DLA tasks. We propose the Key Samples Selection (KSS) method to find key samples in high-level tasks (semantic segmentation) more accurately through agent collaboration, effectively reducing costs. Once selected, these key samples are passed to human beings for active labeling, then the model will be updated with the labeled samples. Hence, we revisited the learning system from reinforcement learning and designed a sample-based agent update strategy, which effectively improves the agent's ability to accept new samples. It achieves significant improvement results in two benchmarks (DSSE-200 (from 77.1% to 86.3%) and CS-150 (from 88.0% to 95.6%)) by using 10% of labeled data.

show abstract

“…Following the setting of the paper [15,64,65,66], we evaluate our model using accuracy, precision, recall and F1 as metrics.…”

Section: Methodsmentioning

confidence: 99%

Human-In-The-Loop Document Layout Analysis

Wu,

Ma,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A few studies have been conducted on the identification of tables in documents [1,2,3,4,5]. However, there is significantly less work put into detecting table structures, and the table structure is frequently classified by the rows and columns of a table [6,7,8].…”

Section: Introductionmentioning

confidence: 99%

“…Deep learning has recently achieving state-of-the-art using convolutional neural network (CNN) [9] in many tasks including object detection [10], face recognition [11], sequence to sequence learning [12,13], speech recognition [14], semantic segmentation [15], image classification [16], handwritten recognition [17,18,19], and table detection [1,8,6] is demanding because they need to classify tables among the texts and other figures. The presence of split columns or rows, as well as nested tables or embedded figures, makes the detection of a table even more difficult.…”

Section: Introductionmentioning

confidence: 99%

TNCR: Table Net Detection and Classification Dataset

Abdallah,

Berendeyev,

Nuradin

et al. 2021

Preprint

View full text Add to dashboard Cite

We present TNCR, a new table dataset with varying image quality collected from free websites. TNCR dataset can be used for table detection in scanned document images and their classification into 5 different classes. TNCR contains 9428 high-quality labeled images. In this paper, we have implemented state-of-the-art deep learning-based methods for table detection to create several strong baselines. Cascade Mask R-CNN with ResNeXt-101-64x4d Backbone Network achieves the highest performance compared to other methods with a precision of 79.7%, recall of 89.8%, and f1 score of 84.4% on the TNCR dataset. We have made TNCR open source in the hope of encouraging more deep learning approaches to table detection, classification and structure recognition. The dataset and trained model checkpoints are available at https://github.com/abdoelsayed2016/TNCR_Dataset

show abstract

“…Additionally, [5] was unable to predict text entities that span multiple text lines, a problem solved with our dynamic graph editing. An alternative formulation to solve form understanding visually is to treat it as a pixel labeling problem, as in Sarkar et al [16]. However, it is not clear from [16] how to infer form structure (bounding boxes and relationships) from pixel predictions, and the proposed (even dilated) CNN model could have difficulty modeling relationships between spatially distant elements.…”

Section: Introductionmentioning

confidence: 99%

“…An alternative formulation to solve form understanding visually is to treat it as a pixel labeling problem, as in Sarkar et al [16]. However, it is not clear from [16] how to infer form structure (bounding boxes and relationships) from pixel predictions, and the proposed (even dilated) CNN model could have difficulty modeling relationships between spatially distant elements. Instead we use a GCN that directly predicts the form structure and does not need to rely on limited receptive fields to propagate information spatially.…”

Section: Introductionmentioning

confidence: 99%

Visual FUDGE: Form Understanding via Dynamic Graph Editing

Davis¹,

Morse²,

Price³

et al. 2021

Preprint

View full text Add to dashboard Cite

We address the problem of form understanding: finding text entities and the relationships/links between them in form images. The proposed FUDGE model formulates this problem on a graph of text elements (the vertices) and uses a Graph Convolutional Network to predict changes to the graph. The initial vertices are detected text lines and do not necessarily correspond to the final text entities, which can span multiple lines. Also, initial edges contain many false-positive relationships. FUDGE edits the graph structure by combining text segments (graph vertices) and pruning edges in an iterative fashion to obtain the final text entities and relationships. While recent work in this area has focused on leveraging large-scale pre-trained Language Models (LM), FUDGE achieves the same level of entity linking performance on the FUNSD dataset by learning only visual features from the (small) provided training set. FUDGE can be applied on forms where text recognition is difficult (e.g. degraded or historical forms) and on forms in resource-poor languages where pre-training such LMs is challenging. FUDGE is stateof-the-art on the historical NAF dataset.

show abstract

Document Structure Extraction Using Prior Based High Resolution Hierarchical Semantic Segmentation

Cited by 11 publications

References 44 publications

Human-In-The-Loop Document Layout Analysis

Human-In-The-Loop Document Layout Analysis

TNCR: Table Net Detection and Classification Dataset

Visual FUDGE: Form Understanding via Dynamic Graph Editing

Contact Info

Product

Resources

About