Example-Based Sketch Segmentation and Labeling Using CRFs

Schneider, Rosalia; Tuytelaars, Tinne

doi:10.1145/2898351

Cited by 56 publications

(40 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above contents will be explained in the appendix. Table II shows that our method outperforms the Huang's method [8] and has similar performance but much less test time-consuming(1 to 2 sketches per second) compared with the CRF model [18]. However, towards certain categories, SFSegNet is about 20% less than CRF.…”

Section: Resultsmentioning

confidence: 77%

“…In order to fit the input sketch into a certain 3D model, their technic needs a part-labeled 3D model repository and a sketch-based shape retrieval to estimate the viewpoint and the category of the input sketch. Similarly, classification before segmentation, Schneider et al [18] adopt Fisher Vectors(FVs) [17] for sketch classification and segment them at points with a high curvature. In consideration of the relations between segments, they fit the segmentation results into a CRF model to encode these relations.…”

Section: B Sketch Segmentation and Labelingmentioning

confidence: 99%

“…As for the dataset, we introduce the instance-level sketch segmentation dataset [25] extended from Huang's benchmark [8], consisting 10,000 annotated sketches, collected by both experts and non-experts depicting objects in ten categories after observing photos or simply imagining. To evaluate the performance of the proposed network architecture, we compare it with state-of-the-art image segmentation method FCN [13], LinkNet-34 [20], U-Net [16] and sketch segmentation method Huang's [8], CRFs [18] on both the introduced dataset and Huang's benchmark.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SFSegNet: Parse Freehand Sketches using Deep Fully Convolutional Networks

Jiang

Wang

Lin

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Parsing sketches via semantic segmentation is attractive but challenging, because (i) free-hand drawings are abstract with large variances in depicting objects due to different drawing styles and skills; (ii) distorting lines drawn on the touchpad make sketches more difficult to be recognized; (iii) the high-performance image segmentation via deep learning technologies needs enormous annotated sketch datasets during the training stage.In this paper, we propose a Sketch-target deep FCN Segmentation Network(SFSegNet) for automatic free-hand sketch segmentation, labeling each sketch in a single object with multiple parts. SFSegNet has an end-to-end network process between the input sketches and the segmentation results, composed of 2 parts: (i) a modified deep Fully Convolutional Network(FCN) using a reweighting strategy to ignore background pixels and classify which part each pixel belongs to; (ii) affine transform encoders that attempt to canonicalize the shaking strokes. We train our network with the dataset that consists of 10,000 annotated sketches, to find an extensively applicable model to segment stokes semantically in one ground truth. Extensive experiments are carried out and segmentation results show that our method outperforms other state-of-the-art networks.

show abstract

Section: Resultsmentioning

confidence: 77%

Section: B Sketch Segmentation and Labelingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SFSegNet: Parse Freehand Sketches using Deep Fully Convolutional Networks

Jiang

Wang

Lin

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…Huang et al [2014] leveraged a repository of 3D template models composed of semantically segmented and labeled components to derive part-level structures. Schneider and Tuytelaars [2016] performed sketch segmentation by looking at salient geometrical features (such as T-junctions and X-junctions) via a Conditional Random Field (CRF) framework. Instead of studying single object recognition or part-level sketch segmentation, we conduct an exploratory study for scene-level parsing of sketches, by using the large-scale scene sketch dataset SketchyScene [Zou et al 2018].…”

Section: Sketch Understandingmentioning

confidence: 99%

Language-based colorization of scene sketches

Zhang

Gao

et al. 2019

ACM Trans. Graph.

View full text Add to dashboard Cite

It's a moonlit night. There is a moon in the sky. A house is in the middle. A car is in front of the house. Three trees are on the left of the house. Two trees are on the right back of the car. There is a road.It is a moonlit night. There is a yellow moon in the sky. An orange house with light gray roof is in the middle. A yellow car with blue window is in front of the house. Three green trees are on the left of the house. Two green trees are on the right back of the car. There is a yellow road. The sky is blue and all things are on green grass.It is a moonlit night. There is a purple moon in the sky. A yellow house with yellow roof is in the middle. A blue car with light gray window is in front of the house. Three yellow trees are on the left of the house. Two yellow trees are on the right back of the car. There is a light brown road. The sky is gray and all things are on yellow grass.It is a moonlit night. There is a yellow moon in the sky. A blue house with blue roof is in the middle. A red car with blue window is in front of the house. Three green trees are on the left of the house. Two green trees are on the right back of the car. There is a dark gray road. The sky is blue and all things are on green grass. Fig. 1. We present LUCSS, a language-based interactive colorization system for scene sketches. It takes advantage of both instance level segmentation and language models in a unified generative adversarial network, allowing users to accomplish different colorization goals in a form of language instructions. Left: input scene sketch and its content description automatically generated by LUCSS. Three right columns show the colorization results generated by LUCSS following three different instructions at the bottom. Texts underlined are user-specified, with the target colors highlighted in bold.Abstract. We introduce LUCSS, a language-based system for interactive colorization of scene sketches, based on their semantic understanding. LUCSS is built upon deep neural networks trained via a large-scale repository of scene sketches and cartoon-style color images with text descriptions. It consists of three sequential modules. First, given a scene sketch, the segmentation module automatically partitions an input sketch into individual object instances. Next, the captioning module generates the text description with spatial relationships based on the instance-level segmentation results. Finally, the interactive colorization module allows users to edit the caption and produce colored images based on the altered caption. Our experiments show the effectiveness of our approach and the desirability of its components to alternative choices.

show abstract

“…There have been prior works that address the semantic segmentation task for sketches, the main challenge being that appearance is sparse. The work of Huang et al [HFL14] uses a learning based approaches, while Schneider and Tuytelaars [ST16] performs a segmentation of sketch regions by looking at features such as curvature point, T junction and X junctions, and then refines the result with a CRF to compute a segmentation. These approaches generate a high quality sketch segmentation, however their task is fundamentally different from ours, in that we are attempting an instance‐segmentation on the stroke level, rather than semantic classification of a stroke or groups of strokes.…”

Section: Related Workmentioning

confidence: 99%

Semantic Segmentation for Line Drawing Vectorization Using Neural Networks

Kim

Wang

Öztireli

et al. 2018

Computer Graphics Forum

View full text Add to dashboard Cite

In this work, we present a method to vectorize raster images of line art. Inverting the rasterization procedure is inherently ill‐conditioned, as there exist many possible vector images that could yield the same raster image. However, not all of these vector images are equally useful to the user, especially if performing further edits is desired. We therefore define the problem of computing an instance segmentation of the most likely set of paths that could have created the raster image. Once the segmentation is computed, we use existing vectorization approaches to vectorize each path, and then combine all paths into the final output vector image. To determine which set of paths is most likely, we train a pair of neural networks to provide semantic clues that help resolve ambiguities at intersection and overlap regions. These predictions are made considering the full context of the image, and are then globally combined by solving a Markov Random Field (MRF). We demonstrate the flexibility of our method by generating results on character datasets, a synthetic random line dataset, and a dataset composed of human drawn sketches. For all cases, our system accurately recovers paths that adhere to the semantics of the drawings.

show abstract

Example-Based Sketch Segmentation and Labeling Using CRFs

Cited by 56 publications

References 20 publications

SFSegNet: Parse Freehand Sketches using Deep Fully Convolutional Networks

SFSegNet: Parse Freehand Sketches using Deep Fully Convolutional Networks

Language-based colorization of scene sketches

Semantic Segmentation for Line Drawing Vectorization Using Neural Networks

Contact Info

Product

Resources

About