READ: Recursive Autoencoders for Document Layout Generation

Patil, Akshay Gadi; Ben‐Eliezer, Omri; Perel, Or; Averbuch‐Elor, Hadar

doi:10.1109/cvprw50498.2020.00280

Cited by 43 publications

(12 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NDN (Lee et al 2020) leverages Graph Convolution Networks (GCNs (Scarselli et al 2008;Kipf and Welling 2016)) to learn the layout representation, where the labels of relationships are based on heuristics (e.g., top, below and larger). Similarly, READ (Patil et al 2020) also uses heuristics to determine relationships between elements and then leverage Recursive Neural Networks (RvNNs (Goller and Kuchler 1996)) for layout generation. These studies impose unreasonable restrictions when generating layout.…”

Section: Related Workmentioning

confidence: 99%

“…To aid the creation of graphic layouts, growing interest has been devoted to automatic layout generation. Most studies abstract the layout into a list of bounding boxes and generate layouts by predicting positions of all the elements in one go (Li et al 2019;Patil et al 2020;Lee et al 2020;Gupta et al 2020;Arroyo, Postels, and Tombari 2021). Specifically, recent work explores generic layout generation by leveraging Transformer, while early studies usually impose restrictions when generating layouts (e.g., using heuristicbased labels for element relationships and handling a lim-Figure 1: Examples for graphic layouts.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Coarse-to-Fine Generative Modeling for Graphic Layouts

Jiang

Sun

Zhu

et al. 2022

AAAI

View full text Add to dashboard Cite

Even though graphic layout generation has attracted growing attention recently, it is still challenging to synthesis realistic and diverse layouts, due to the complicated element relationships and varied element arrangements. In this work, we seek to improve the performance of layout generation by incorporating the concept of regions, which consist of a smaller number of elements and appears like a simple layout, into the generation process. Specifically, we leverage Variational Autoencoder (VAE) as the overall architecture and decompose the decoding process into two stages. The first stage predicts representations for regions, and the second stage fills in the detailed position for each element within the region based on the predicted region representation. Compared to prior studies that merely abstract the layout into a list of elements and generate all the element positions in one go, our approach has at least two advantages. First, by the two-stage decoding, our approach decouples the complex layout generation task into several simple layout generation tasks, which reduces the problem difficulty. Second, the predicted regions can help the model roughly know what the graphic layout looks like and serve as global context to improve the generation of detailed element positions. Qualitative and quantitative experiments demonstrate that our approach significantly outperforms the existing methods, especially on the complex graphic layouts.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Coarse-to-Fine Generative Modeling for Graphic Layouts

Jiang

Sun

Zhu

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…Document Intelligence can be considered as an umbrella term covering problems of Key Information Extraction [10,54], Table Detection [41,38] and Structure Recognition [39,55], Document Layout Segmentation [5,4] Document Layout Generation [6,36,3,48], Document Visual Question Answering [51,50,32], Document Image Enhancement [49,22,47] which involves the understanding of visually rich semantic information and structure of different layout entities of a whole page.…”

Section: Related Workmentioning

confidence: 99%

OCR-IDL: OCR Annotations for Industry Document Library Dataset

Biten¹,

Tito²,

Gómez³

et al. 2022

Preprint

View full text Add to dashboard Cite

Pretraining has proven successful in Document Intelligence tasks where deluge of documents are used to pretrain the models only later to be finetuned on downstream tasks. One of the problems of the pretraining approaches is the inconsistent usage of pretraining data with different OCR engines leading to incomparable results between models. In other words, it is not obvious whether the performance gain is coming from diverse usage of amount of data and distinct OCR engines or from the proposed models. To remedy the problem, we make public the OCR annotations for IDL documents using commercial OCR engine given their superior performance over open source OCR models. The contributed dataset (OCR-IDL) has an estimated monetary value over 20K US$. It is our hope that OCR-IDL can be a starting point for future works on Document Intelligence. All of our data and its collection process with the annotations can be found in https://github.com/furkanbiten/idl_data.

show abstract

“…al. [13] has come up with a solution called 'READ' that can make use of this highly structured positional information along with content to generate document layouts. Their recursive neural network-based resulting model architecture provided state-of-the-art results for generating synthetic layouts for 2D documents.…”

Section: Related Workmentioning

confidence: 99%

DocSynth: A Layout Guided Approach for Controllable Document Image Synthesis

Biswas¹,

Riba²,

Lladós³

et al. 2021

Preprint

View full text Add to dashboard Cite

Despite significant progress on current state-of-the-art image generation models, synthesis of document images containing multiple and complex object layouts is a challenging task. This paper presents a novel approach, called DocSynth, to automatically synthesize document images based on a given layout. In this work, given a spatial layout (bounding boxes with object categories) as a reference by the user, our proposed DocSynth model learns to generate a set of realistic document images consistent with the defined layout. Also, this framework has been adapted to this work as a superior baseline model for creating synthetic document image datasets for augmenting real data during training for document layout analysis tasks. Different sets of learning objectives have been also used to improve the model performance. Quantitatively, we also compare the generated results of our model with real data using standard evaluation metrics. The results highlight that our model can successfully generate realistic and diverse document images with multiple objects. We also present a comprehensive qualitative analysis summary of the different scopes of synthetic image generation tasks. Lastly, to our knowledge this is the first work of its kind.

show abstract

READ: Recursive Autoencoders for Document Layout Generation

Cited by 43 publications

References 23 publications

Coarse-to-Fine Generative Modeling for Graphic Layouts

Coarse-to-Fine Generative Modeling for Graphic Layouts

OCR-IDL: OCR Annotations for Industry Document Library Dataset

DocSynth: A Layout Guided Approach for Controllable Document Image Synthesis

Contact Info

Product

Resources

About