Free-Hand Sketch Synthesis with Deformable Stroke Models

Li, Yi; Song, Yi-Zhe; Hospedales, Timothy M.; Gong, Shaogang

doi:10.1007/s11263-016-0963-9

Cited by 43 publications

(26 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Photo-to-sketch is extremely challenging due to the large domain gap and the fact that the sketch domain is generated by humans with variable drawing styles. As a result, only sketch-to-photo synthesis has been studied so far [36,20,29]. In this work, we study photo-to-sketch synthesis with the novel approach of treating sketch generation as a photo-to-sketch abstraction process.…”

Section: Related Workmentioning

confidence: 99%

Learning Deep Sketch Abstraction

Muhammad

Yang

Song

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and finegrained sketch-based image retrieval (FG-SBIR). A fundamental challenge for sketch analysis is to deal with drastically different human drawing styles, particularly in terms of abstraction level. In this work, we propose the first stroke-level sketch abstraction model based on the insight of sketch abstraction as a process of trading off between the recognizability of a sketch and the number of strokes used to draw it. Concretely, we train a model for abstract sketch generation through reinforcement learning of a stroke removal policy that learns to predict which strokes can be safely removed without affecting recognizability. We show that our abstraction model can be used for various sketch analysis tasks including: (1) modeling stroke saliency and understanding the decision of sketch recognition models, (2) synthesizing sketches of variable abstraction for a given category, or reference object instance in a photo, and (3) training a FG-SBIR model with photos only, bypassing the expensive photo-sketch pair collection step.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Deep Sketch Abstraction

Muhammad

Yang

Song

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

show abstract

“…Segmenting sketches on the semantic level could facilitate applications like sketch-based 3D shape retrieval [5], scene modeling [8], part assembly [20] and free-hand sketch synthesis [9]. And most early study on this topic rely on handcrafted features with various models from the field of machine learning.…”

Section: Sketch Segmentationmentioning

confidence: 99%

Multi-column point-CNN for sketch segmentation

Wang

Lin

et al. 2020

Neurocomputing

View full text Add to dashboard Cite

Traditional sketch segmentation methods mainly rely on handcrafted features and complicate models, and their performance is far from satisfactory due to the abstract representation of sketches. Recent success of Deep Neural Networks (DNNs) in related tasks suggests DNNs could be a practical solution for this problem, yet the suitable datasets for learning and evaluating DNNs are limited. To this end, we introduce SketchSeg, a large dataset consisting of 10,000 pixel-wisely labeled sketches. Besides, due to the lack of colors and textures in sketches, conventional DNNs learned on natural images are not optimal for tackling our problem. Therefore, we further propose the Multi-column Point-CNN (MCPNet), which (1) directly takes sampled points as its input to reduce computational costs, and (2) adopts multiple columns with different filter sizes to better capture the structures of sketches. Extensive experiments validate that the MCPNet is superior to conventional DNNs like FCN. The SketchSeg dataset is publicly available on https://drive.google.com/open? id=1OpCBvkInhxvfAHuVs-spDEppb8iXFC3C.

show abstract

“…is limitation requires the application of uniform attributes to supercategorize and form subcategories that create branching processing and become ineffective. Li et al [21] also presented another freehand sketch synthesis approach using deformable stroke model which consists of standard drawing format and varying formats of each shape and symbol. Based on these strokes' knowledge, the generative data-driven model detects the diverse sketch objects without any training or additional alignments.…”

Section: Related Workmentioning

confidence: 99%

SKETRACK: Stroke-Based Recognition of Online Hand-Drawn Sketches of Arrow-Connected Diagrams and Digital Logic Circuit Diagrams

Altun

Nooruldeen

2019

Scientific Programming

View full text Add to dashboard Cite

Digitalization of handwritten documents has created a greater need for accurate online recognition of hand-drawn sketches. However, the online recognition of hand-drawn diagrams is an enduring challenge in human-computer interaction due to the complexity in extracting and recognizing the visual objects reliably from a continuous stroke stream. This paper focuses on the design and development of a new, efficient stroke-based online hand-drawn sketch recognition scheme named SKETRACK for hand-drawn arrow diagrams and digital logic circuit diagrams. The fundamental parts of this model are text separation, symbol segmentation, feature extraction, classification, and structural analysis. The proposed scheme utilizes the concepts of normalization and segmentation to isolate the text from the sketches. Then, the features are extracted to model different structural variations of the strokes that are categorized into the arrows/lines and the symbols for effective processing. The strokes are clustered using the spectral clustering algorithm based on p-distance and Euclidean distance to compute the similarity between the features and minimize the feature dimensionality by grouping similar features. Then, the symbol recognition is performed using modified support vector machine (MSVM) classifier in which a hybrid kernel function with a lion optimized tuning parameter of SVM is utilized. Structural analysis is performed with lion-based task optimization for recognizing the symbol candidates to form the final diagram representations. This proposed recognition model is suitable for simpler structures such as flowcharts, finite automata, and the logic circuit diagrams. Through the experiments, the performance of the proposed SKETRACK scheme is evaluated on three domains of databases and the results are compared with the state-of-the-art methods to validate its superior efficiency.

show abstract

Free-Hand Sketch Synthesis with Deformable Stroke Models

Cited by 43 publications

References 42 publications

Learning Deep Sketch Abstraction

Learning Deep Sketch Abstraction

Multi-column point-CNN for sketch segmentation

SKETRACK: Stroke-Based Recognition of Online Hand-Drawn Sketches of Arrow-Connected Diagrams and Digital Logic Circuit Diagrams

Contact Info

Product

Resources

About