Deep Sketch-Based Modeling: Tips and Tricks

Zhong, Yan; Gryaditskaya, Yulia; Zhang, Honggang; Song, Yi-Zhe

doi:10.1109/3dv50981.2020.00064

Cited by 29 publications

(15 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sketch-based modeling is a problem that has been studied for a long time. The earlier methods predicted local geometric properties from hand-crafted rules and then inferred the 3D shape from the geometric properties [33,32]. In recent years, some deep learning based methods have been proposed for sketch-based 3D modeling.…”

Section: Deep Sketch-based 3d Modelingmentioning

confidence: 99%

“…To alleviate the deformation of sketches, they proposed a sketch standardization module to alleviate the deformation problem of sketches. The work [32] discussed the additional challenges of line drawings in comparison with images in 3D reconstruction. In [33], sketches from two viewpoints were used as the input to perform 3D reconstruction, and they collected two datasets, ProSketch and AmateurSketch.…”

Section: Deep Sketch-based 3d Modelingmentioning

confidence: 99%

See 1 more Smart Citation

SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling

Gao¹,

Qian²,

Shen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape. Existing works try to employ the global feature extracted from sketch to directly predict the 3D coordinates, but they usually suffer from losing fine details that are not faithful to the input sketch. Through analyzing the 3D-to-2D projection process, we notice that the density map that characterizes the distribution of 2D point clouds(i.e., the probability of points projected at each location of the projection plane) can be used as a proxy to facilitate the reconstruction process. To this end, we first translate a sketch via an image translation network to a more informative 2D representation that can be used to generate a density map. Next, a 3D point cloud is reconstructed via a two-stage probabilistic sampling process: first recovering the 2D points(i.e., the x and y coordinates) by sampling the density map; and then predicting the depth(i.e., the z coordinate) by sampling the depth values at the ray determined by each 2D point. Extensive experiments are conducted, and both quantitative and qualitative results show that our proposed approach significantly outperforms other baseline methods.

show abstract

Section: Deep Sketch-based 3d Modelingmentioning

confidence: 99%

Section: Deep Sketch-based 3d Modelingmentioning

confidence: 99%

SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling

Gao¹,

Qian²,

Shen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Shape and Appearance Reconstruction. Extensive works have explored the problem of 3D reconstruction from different modalities, such as RGB images [27,11], videos [59], sketches [26,20,64,63], or even text [8]. This problem has also been explored under diverse representations [11,15,35,54,14,40,9,37,50,58] and different levels of supervision [11,15,27,16,59].…”

Section: Related Workmentioning

confidence: 99%

Cross-Modal 3D Shape Generation and Manipulation

Cheng¹,

Chai²,

Ren³

et al. 2022

Preprint

View full text Add to dashboard Cite

Creating and editing the shape and color of 3D objects require tremendous human effort and expertise. Compared to direct manipulation in 3D interfaces, 2D interactions such as sketches and scribbles are usually much more natural and intuitive for the users. In this paper, we propose a generic multi-modal generative model that couples the 2D modalities and implicit 3D representations through shared latent spaces. With the proposed model, versatile 3D generation and manipulation are enabled by simply propagating the editing from a specific 2D controlling modality through the latent spaces. For example, editing the 3D shape by drawing a sketch, re-colorizing the 3D surface via painting color scribbles on the 2D rendering, or generating 3D shapes of a certain category given one or a few reference images. Unlike prior works, our model does not require re-training or fine-tuning per editing task and is also conceptually simple, easy to implement, robust to input domain shifts, and flexible to diverse reconstruction on partial 2D inputs. We evaluate our framework on two representative 2D modalities of grayscale line sketches and rendered color images, and demonstrate that our method enables various shape manipulation and generation tasks with these 2D modalities.

show abstract

“…Each of them respectively sets meshes, voxels, implicit functions, and point clouds as a 3D shape representation. For a fair comparison, we trained the four models with synthetic sketches from [Zhong et al 2020a] and then tested them on 300 pairs of novice and professional sketches to obtain 600 reconstructed models for each method.…”

Section: Preliminary Evaluation Of Sketch-based 3d Reconstructionmentioning

confidence: 99%

DifferSketching: How Differently Do People Sketch 3D Objects?

Xiao¹,

Su²,

Liao³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

the existing datasets. We analyze the collected data at three levels, i.e., sketchlevel, stroke-level, and pixel-level, under both spatial and temporal characteristics, and within and across groups of creators. We found that the drawings by professionals and novices show significant differences at stroke-level, both intrinsically and extrinsically. We demonstrate the usefulness of our dataset in two applications: (i) freehand-style sketch synthesis, and (ii) posing it as a potential benchmark for sketch-based 3D reconstruction. Our dataset and code are available at https://chufengxiao.github.io/DifferSketching/.CCS Concepts: • Applied computing → Arts and humanities; • Humancentered computing → Interaction design; User studies; • Computing methodologies → Graphics systems and interfaces.

show abstract

Deep Sketch-Based Modeling: Tips and Tricks

Cited by 29 publications

References 26 publications

SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling

SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling

Cross-Modal 3D Shape Generation and Manipulation

DifferSketching: How Differently Do People Sketch 3D Objects?

Contact Info

Product

Resources

About