DeepSketchHair: Deep Sketch-Based 3D Hair Modeling

Abstract

We present DeepSketchHair, a deep learning based tool for interactive modeling of 3D hair from 2D sketches. Given a 3D bust model as reference, our sketching system takes as input a user-drawn sketch (consisting of hair contour and a few strokes indicating the hair growing direction within a hair region), and automatically generates a 3D hair model, which matches the input sketch both globally and locally. The key enablers of our system are two carefully designed neural networks, namely, S2ONet, which converts an input sketch to a dense 2D hair orientation field; and O2VNet, which maps the 2D orientation field to a 3D vector field. Our system also supports hair editing with additional sketches in new views. This is enabled by another deep neural network, V2VNet, which updates the 3D vector field with respect to the new sketches. All the three networks are trained with synthetic data generated from a 3D hairstyle database. We demonstrate the effectiveness and expressiveness of our tool using a variety of hairstyles and also compare our method with prior art.Index Terms-Sketch-based hair modeling, 3D volumetric structure, deep learning, generative adversarial networks !

NeuralHDHair: Automatic High-fidelity Hair Modeling from a Single Image Using Implicit Neural Representations

“…Data. Similar to the previous research [6,31,36], we collected 653 3D hair-strand models and aligned them to the same bust model within a boundary box. In addition, we also augment the data by horizontally flipping, scaling, and rotating.…”

“…Second, the existing methods for inferring 3D orientation fields are either time-consuming due to the use of a complex searching and matching process based on a large hair dataset [6,11], or liable to over-smoothness due to the use of deep networks to directly achieve image-to-voxel inference [36]. Third, the conventional hair growth algorithm [31,39] to extract strands from the estimated 3D orientation field are inefficient and not conducive to one-shot hair modeling. Although Zhou et al [40] have attempted to ignore the hair growth procedure by directly regressing 3D hair strands, their reconstruction results are generally unsatisfactory (see Sec.…”

SketchHairSalon: Deep Sketch-based Hair Image Synthesis

“…Hair modeling has been extensively explored in the computer graphics community. Most previous works have focused on reconstructing 3D hair models from either real images [Jakob et al 2009;Wei et al 2005;Zhang et al 2018; or userspecified sketches [Fu et al 2007;Hu et al 2015;Mao et al 2004;Shen et al 2020]. For example, given a single-view hair image for hair modeling, Chai et al [2013] utilize a few strokes to guide hair directions to reduce the ambiguity of growing hair strands from the hair image.…”

“…Due to the requirement of 3D inputs, it is difficult to apply their technique to our problem. Shen et al [2020] introduce a deep learning based framework for strand-level hair modeling based on 2D sketches to produce plausible 3D hairstyles. Although their method can generate high-quality results with realistic appearance and layering effects to some extent, mainly due to the use of 2D and 3D orientation fields as intermediate hair representations, their method is not capable of modeling hairstyles with complex structures like braided hairstyles.…”

HAO‐CNN: Filament‐aware hair reconstruction based on volumetric vector fields

“…Yang et al further extended the Hair‐GANs to capture dynamic hair geometries from monocular videos by designing a HairSpatNet and a HairTempNet. DeepSketchHair 16 interactively reconstruct hair geometries by mapping the strokes and contours to a 2D dense orientation map, and then generates a hair VVF. Unfortunately, these VVF‐based methods are memory expensive if increasing volumetric resolution and time consuming due to involving 3D convolution.…”