Online Handwritten Diagram Recognition with Graph Attention Networks

Yun, Xiao-Long; Zhang, Yanming; Ye, Jun-Yu; Liu, Cheng‐Lin

doi:10.1007/978-3-030-34120-6_19

Cited by 9 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dataset is publicly available, and its size has been increased to 419 flowcharts after the publication date. Following the release, several methods for online flowchart recognition were proposed [2][3][4][5][6]9,18,36,37,40]. Wu et al [38] is the first work that uses FC_A for offline recognition.…”

Section: Handwritten Diagram Recognitionmentioning

confidence: 99%

“…This input device captures the drawing as a temporal sequence of strokes. Online diagram recognition has received a lot of attention in research, especially in the area of flowcharts [1][2][3][4][5][6]9,14,18,36,37,40]. Yet, those approaches are of limited applicability if the original stroke data are not available (e.g., hand-drawn diagrams on paper).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Arrow R-CNN for handwritten diagram recognition

Schäfer

Keuper

Stuckenschmidt

2021

IJDAR

View full text Add to dashboard Cite

We address the problem of offline handwritten diagram recognition. Recently, it has been shown that diagram symbols can be directly recognized with deep learning object detectors. However, object detectors are not able to recognize the diagram structure. We propose Arrow R-CNN, the first deep learning system for joint symbol and structure recognition in handwritten diagrams. Arrow R-CNN extends the Faster R-CNN object detector with an arrow head and tail keypoint predictor and a diagram-aware postprocessing method. We propose a network architecture and data augmentation methods targeted at small diagram datasets. Our diagram-aware postprocessing method addresses the insufficiencies of standard Faster R-CNN postprocessing. It reconstructs a diagram from a set of symbol detections and arrow keypoints. Arrow R-CNN improves state-of-the-art substantially: on a scanned flowchart dataset, we increase the rate of recognized diagrams from 37.7 to 78.6%.

show abstract

Section: Handwritten Diagram Recognitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Arrow R-CNN for handwritten diagram recognition

Schäfer

Keuper

Stuckenschmidt

2021

IJDAR

View full text Add to dashboard Cite

show abstract

“…[1] and LSTMs have been shown to be quite successful [4]. Recently, [18] have applied graph attention networks to the 1,300 diagrams from [13,16,17] for text/non-text classification using a handengineered stroke feature vector. Interestingly, the method outperforms [17] only by a small margin.…”

Section: Related Workmentioning

confidence: 99%

CoSE: Compositional Stroke Embeddings

Aksan,

Deselaers,

Tagliasacchi

et al. 2020

Preprint

View full text Add to dashboard Cite

We present a generative model for stroke-based drawing tasks which is able to model complex free-form structures. While previous approaches rely on sequencebased models for drawings of basic objects or handwritten text, we propose a model that treats drawings as a collection of strokes that can be composed into complex structures such as diagrams (e.g., flow-charts). At the core of the approach lies a novel auto-encoder that projects variable-length strokes into a latent space of fixed dimension. This representation space allows a relational model, operating in latent space, to better capture the relationship between strokes and to predict subsequent strokes. We demonstrate qualitatively and quantitatively that our proposed approach is able to model the appearance of individual strokes, as well as the compositional structure of larger diagram drawings. Our approach is suitable for interactive use cases such as auto-completing diagrams.

show abstract

Generative Ink: Data-Driven Computational Models for Digital Ink

Aksan

Hilliges

2021

Human–Computer Interaction Series

View full text Add to dashboard Cite

Online Handwritten Diagram Recognition with Graph Attention Networks

Cited by 9 publications

References 17 publications

Arrow R-CNN for handwritten diagram recognition

Arrow R-CNN for handwritten diagram recognition

CoSE: Compositional Stroke Embeddings

Generative Ink: Data-Driven Computational Models for Digital Ink

Contact Info

Product

Resources

About