A Generative Model of People in Clothing

Abstract

Figure 1: Random examples of people generated with our model. For each row, sampling is conditioned on the silhouette displayed on the left. Our proposed framework also supports unconditioned sampling as well as conditioning on local appearance cues, such as color. AbstractWe present the first image-based generative model of people in clothing for the full body. We sidestep the commonly used complex graphics rendering pipeline and the need for high-quality 3D scans of dressed people. Instead, we learn generative models from a large image database. The main challenge is to cope with the high variance in human pose, shape and appearance. For this reason, pure image-based approaches have not been considered so far. We show that this challenge can be overcome by splitting the generating process in two parts. First, we learn to generate a semantic segmentation of the body and clothing. Second, we learn a conditional model on the resulting segments that creates realistic images. The full model is differentiable and can be conditioned on pose, shape or color. The result are samples of people in different clothing items and styles. The proposed model can generate entirely new people with realistic clothing. In several experiments we present encouraging results that suggest an entirely data-driven approach to people generation is possible. * This work was performed while P. V. Gehler was with the BCCN 1 and MPI-IS 2 .

Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation

“…Consequently, it is not clear -despite excellent performance on standard benchmarks -how methods [59,25,37,26] generalize to in-the-wild images. To add variation, some methods resort to generating synthetic images [46,64,23] but it is complex to approximate fully realistic images with sufficient variance. Similar to model-based methods, learning approaches have benefited from the advent of robust 2D pose methods -by matching 2D detections to a 3D pose database [8,66], by regressing pose from 2D joint distance matrices [35], by exploiting pose and geometric priors for lifting [69,1,51,19,32,70,47]; or simply by training a feed forward network to directly predict 3D pose from 2D joints [30].…”

Dense Intrinsic Appearance Flow for Human Pose Transfer

“…Generating human-centric images is an important sub-area of image synthesis. Example tasks range from generating full human body in clothing [18] to generating human action sequences [3]. Ma et al [23] are the first ones to approach the task of human pose transfer, which aims to generate a person image in a target pose if a reference image of that person is given beforehand.…”

Multi-Garment Net: Learning to Dress 3D People From Images

“…Since current statistical models can not represent clothing, most works [7,26,40,68,48,32,22,67,31,50,8,33,44,46] are restricted to inferring body shape alone. Model fits have been used to virtually dress and manipulate people's shape and clothing in images [50,67,62,36]. None of these approaches recover 3D clothing.…”