ClothCap

Abstract

Fig. 1. ClothCap. From left to right: (1) An example 3D textured scan that is part of a 4D sequence. (2) Our multi-part aligned mesh model, layered over the body. (3) The estimated minimally clothed shape (MCS) under the clothing. (4) The body made fatter and dressed in the same clothing. Note that the clothing adapts in a natural way to the new body shape. (5) This new body shape posed in a new, never seen, pose. This illustrates how ClothCap supports a range of applications related to clothing capture, modeling, retargeting, reposing, and try-on.Designing and simulating realistic clothing is challenging. Previous methods addressing the capture of clothing from 3D scans have been limited to single garments and simple motions, lack detail, or require specialized texture patterns. Here we address the problem of capturing regular clothing on fully dressed people in motion. People typically wear multiple pieces of clothing at a time. To estimate the shape of such clothing, track it over time, and render it believably, each garment must be segmented from the others and the body. Our ClothCap approach uses a new multi-part 3D model of clothed bodies, automatically segments each piece of clothing, estimates the minimally clothed body shape and pose under the clothing, and tracks the 3D deformations of the clothing over time. We estimate the garments and their motion from 4D scans; that is, high-resolution 3D scans of the subject in motion at 60 fps. ClothCap is able to capture a clothed person in motion, extract their clothing, and retarget the clothing to new body shapes; this provides a step towards virtual try-on.

Learning‐Based Animation of Clothing for Virtual Try‐On

“…In Figure 8, we show the clothing deformations produced by our approach on a static pose while changing the body shape over time. We compare results with a physics-based simulation and with our implementation of retargeting techniques [LCT18,PMPHB17]. Notice how our method successfully reproduces ground-truth deformations, including the overall drape (i.e., how the T-shirt slides up the belly due to the stretch caused by the increasingly fat character) and mid-scale wrinkles.…”

“…This allows editing the actor's shape and pose parameters while keeping the same captured garment or even changing it. However, re-animated motions lack realism since they cannot predict the nonrigid behavior of clothing under unseen poses or shapes, and are usually limited to copying wrinkles across bodies of different shapes [PMPHB17,LCT18].…”

“…We depict the per-vertex mean error on a static pose (top) and a dynamic sequence (bottom), as we change the body shape over time. To provide a quantitative comparison to existing methods, we additionally show the error suffered by our implementation of cloth retargeting [LCT18,PMPHB17]. As discussed in Section 2, such retargeting methods scale the garment in a way analogous to the body to retain the garment's style.…”

“…In Figure 10, we compare our model to ClothCap [PMPHB17]. Admittedly, the virtual try-on scenario considered by Pons-Moll and colleagues differs from ours: while we assume that the virtual garment is provided by a brand and the customer tests the fit of a certain sized garment on a virtual avatar of his/her body, Pons-Moll et al reconstruct the clothing and body shape from 4D scans, and transfer the captured garment to different shapes.…”

“…In this paper, we propose a learning-based method for cloth animation that meets the needs of virtual try-on, as it models the deformation of a given garment as a function of body motion and shape. Other methods that account for changes in body shape do not deform the garment in a realistic way, and either resize the garment while preserving its style [GRH * 12, BSBC12], or retarget cloth wrinkles to bodies of different shapes [PMPHB17,LCT18].…”