In Good Shape: Robust People Detection based on Appearance and Shape

Abstract

Robustly detecting people in real world scenes is a fundamental and challenging task in computer vision. State-of-the-art approaches use powerful learning methods and manually annotated image data. Importantly, these learning based approaches rely on the fact that the collected training data is representative of all relevant variations necessary to detect people. Rather than to collect and annotate ever more training data, this paper explores the possibility to use a 3D human shape and pose model from computer graphics to add relevant shape information to learn more powerful people detection models. By sampling from the space of 3D shapes we are able to control data variability while covering the major shape variations of humans which are often difficult to capture when collecting real-world training images. We evaluate our data generation method for a people detection model based on pictorial structures. As we show on a challenging multi-viewpoint dataset, the additional information contained in the 3D shape model helps to outperform models trained on image data alone (see e.g. Fig. 1).

Building statistical shape spaces for 3D human modeling

“…Furthermore, they have been used to estimate body shapes from images [5] and 3D scans [15,39] of dressed subjects. Given a 3D body shape, statistical shape spaces can be used to modify input images [41] or videos [18], to automatically generate training sets for people detection [24,23], or to simulate clothing on people [12].…”

“…We extensively evaluate the improved accuracy and generality of our new model, and show its improved performance for human body reconstruction from sparse input data.2). Our model is based on a simplified and efficient variant of the SCAPE model [3] (henceforth termed S-SCAPE space) that was described by Jain et al [18] and used for different applications in computer vision and graphics [18,24,23,17,20], but was never learned from such a complete dataset. This compact shape space learns a probability distribution from a dataset of 3D human laser scans.…”

Novel Views of Objects from a Single Image

“…In contrast, we take an image-based approach in order to leverage real-world image statistics as well as the intra-class variation available to us in image data. Previous image-based work synthesizes training images by recombining poses and appearance of objects in order to create new instances [39], [40], [41], [42], [43]. More recently, Su et al [44] and Peng et al [45] rendered 3D models to generate additional data for object detection and pose estimation respectively using Convolutional Neural Networks.…”

Image-Based Synthesis and Re-synthesis of Viewpoints Guided by 3D Models

“…In contrast, we take an image-based approach in order to leverage real-world image statistics as well as the intra-class variation available to us in image data. Previous image-based work synthesizes training images by recombining poses and appearance of objects in order to create new instances [8,28,29,27,38]. In contrast, our work focuses on synthesis across viewpoints and deals with disocclusions that are not addressed in previous work.…”