Combining geometry and domain knowledge to interpret hand-drawn diagrams

Gennari, Leslie; Kara, Levent Burak; Stahovich, Thomas F.; Shimada, Kenji

doi:10.1016/j.cag.2005.05.007

Cited by 90 publications

(58 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sketch prediction is a challenging problem which depends on sketch recognition [Gennari et al 2005;Hammond and Davis 2004;LaViola Jr. and Zeleznik 2007;Ouyang and Davis 2007;Peterson et al 2010;Sezgin and Davis 2005] since it needs to interpret the incomplete user sketch and discriminate between object classes. In general, [Tirkaz et al 2012] proposes a method by learning visual appearances of partial drawings through semi-supervised clustering, followed by a supervised classification step that determines object classes.…”

Section: Related Workmentioning

confidence: 99%

Mixed heuristic search for sketch prediction on chemical structure drawing

Kang

LaViola

2014

Proceedings of the 4th Joint Symposium on Computational Aesthetics, Non-Photorealistic Animation and Rendering, and Sketch-Base

View full text Add to dashboard Cite

Sketching is a natural way to input chemical structures that can be used to query information from a large chemical structure database. Based on a user's incomplete sketch of a chemical structure, sketch prediction becomes a challenging problem not only due to arbitrary drawings orders among users but also similarities among chemical structure layouts. In this paper, we present a graph-based approach to handle the sketch prediction problem. We use multisets as the data representation of hand-drawn chemical structures and create an undirected graph to handle data in all multisets. This approach transforms the sketch prediction problem into a search problem to find a hamiltonian path in the corresponding sub-graph with polynomial time complexity. We introduce mixed heuristics to guide the search procedure. Through an initial experiment on a hand-drawn chemical structure dataset, we demonstrate that in comparison with a baseline method, the proposed approach improves the prediction accuracy and efficiently predicts chemical structures from only partially sketched drawings.

show abstract

Section: Related Workmentioning

confidence: 99%

Mixed heuristic search for sketch prediction on chemical structure drawing

Kang

LaViola

2014

Proceedings of the 4th Joint Symposium on Computational Aesthetics, Non-Photorealistic Animation and Rendering, and Sketch-Base

View full text Add to dashboard Cite

show abstract

“…During the inference process the system chooses the best candidate for each segment and adds the candidate to the set of final symbol detections. For example, if the system decides that the correct label for y c,2 (the candidate node for segment 2) is a "wedge" candidate containing segments [1,2,4], then the "wedge" candidate is added to the final symbol detections. Note that the candidate node labels can contain multiple interpretations of each candidate, so y c,2 also has "hash" and "text" versions of candidate [1,2,4] as possible labels (the "bond" label is only applied to single-segment candidates).…”

Section: Joint Graphical Model Classifiermentioning

confidence: 99%

“…For example, if the system decides that the label of y c,2 (the candidate node for segment 2) is a "wedge" candidate that spans segments [1,2,4], then the labels for y c,1 and y c,4 also need to be assigned to the same "wedge" candidate.…”

Section: Joint Graphical Model Classifiermentioning

confidence: 99%

“…There is a growing body of sketch recognition research covering a variety of different domains, including electrical circuits [4,13,19], general charts [22,23], and mathematical expressions [9]. Many of these approaches focus on the relationships between geometric primitives like lines, arcs, and curves, specifying them either manually [1,5,6] or learning them from labeled data [19].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

ChemInk

Ouyang

Davis

2011

Proceedings of the 16th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

We describe a new sketch recognition framework for chemical structure drawings that combines multiple levels of visual features using a jointly trained conditional random field. This joint model of appearance at different levels of detail makes our framework less sensitive to noise and drawing variations, improving accuracy and robustness. In addition, we present a novel learning-based approach to corner detection that achieves nearly perfect accuracy in our domain. The result is a recognizer that is better able to handle the wide range of drawing styles found in messy freehand sketches. Our system handles both graphics and text, producing a complete molecular structure as output. It works in real time, providing visual feedback about the recognition progress. On a dataset of chemical drawings our system achieved an accuracy rate of 97.4%, an improvement over the best reported results in literature. A preliminary user study also showed that participants were on average over twice as fast using our sketch-based system compared to ChemDraw, a popular CAD-based tool for authoring chemical diagrams. This was the case even though most of the users had years of experience using ChemDraw and little or no experience using Tablet PCs.

show abstract

“…Most of the approaches described in the literature ( [5], [7], [8], [9]) attempt to solve this problem by requiring the user to provide additional information. However, humans are able to segment sketches without requiring such extra information.…”

Section: Motivationmentioning

confidence: 99%

Human Perception in Segmentation of Sketches

Varley

Piquer

Vergara

et al. 2010

Graphics Recognition. Achievements, Challenges, and Evolution

View full text Add to dashboard Cite

Abstract. In this paper, we study the segmentation of sketched engineering drawings into a set of straight and curved segments. Our immediate objective is to produce a benchmarking method for segmentation algorithms. The criterion is to minimise the differences between what the algorithm detects and what human beings perceive. We have created a set of sketched drawings and have asked people to segment them. By analysis of the produced segmentations, we have obtained the number and locations of the segmentation points which people perceive. Evidence collected during our experiments supports useful hypotheses, for example that not all kinds of segmentation points are equally difficult to perceive. The resulting methodology can be repeated with other drawings to obtain a set of sketches and segmentation data which could be used as a benchmark for segmentation algorithms, to evaluate their capability to emulate human perception of sketches.Keywords: Sketch recognition. Low level ink processing and pen stroke segmentation. Engineering Graphics. Segmentation Ability. PresentationOur interest is computer-based recognition of sketched engineering drawings, such as would allow automated conversion of engineering sketches into CAD representations. Segmentation of the drawing is a critical stage, and one which has received much attention over the years. Some important aspects of segmentation still remain unsolved, perhaps because (as [1] shows), segmentation is not, in fact, a single problem, but a set of similar problems. In this paper, we consider one such unsolved aspect: the benchmarking of computer-based segmentation of sketches. Recognition of the object portrayed in engineering drawings is a topical subfield of graphics recognition, which deals more generally with how computers can interpret semi-structured drawings which contain both freeform elements and symbols defined by convention. In the case of creating 2D or 3D CAD models of engineering objects from single or multiple drawings, it is the freeform elements, lines and curves, which portray the surfaces of the object, and it is these which we wish to identify. At this stage of processing, the conventional symbols (like dimensions and hatching) are clutter, and should be removed (and perhaps stored for later use). Our objective is thus to segment engineering drawings into lines, curves and clutter.When evaluating new segmentation approaches, one common strategy is simply comparing the number of segmentation points obtained by the new approach with the number of segmentation points which the "theoretical" shape possesses (by "theoretical" we mean the ideal primitives obtained from a line drawing by applying a well-defined set of topological and geometrical constraints). This strategy assumes that the new approach should detect those properties which the theoretical shape should possess, regardless of whether or not the actual drawing used as input really does possess them.In reality, we cannot assume that a sketched line drawing on paper will always contain exact...

show abstract

Combining geometry and domain knowledge to interpret hand-drawn diagrams

Cited by 90 publications

References 10 publications

Mixed heuristic search for sketch prediction on chemical structure drawing

Mixed heuristic search for sketch prediction on chemical structure drawing

ChemInk

Human Perception in Segmentation of Sketches

Contact Info

Product

Resources

About