No abstract

We present an approach to capture the 3D motion of a group of people engaged in a social interaction, where inter-occlusions are frequent and functional. The Panoptic Studio is a system organized around the thesis that social interactions should be measured through the integration of perceptual analyses over a large variety of viewpoints. We present a modularized system designed around this principle, consisting of integrated structural, hardware, and software innovations. The system takes, as input, 480 synchronized video streams of multiple people engaged in social activities, and produces, as output, the labeled time-varying 3D structure of anatomical landmarks on individuals in the space. Our algorithm is designed to fuse the "weak" perceptual processes in the large number of views by progressively generating skeletal proposals from low-level appearance cues, and a framework for temporal refinement is also presented by associating body parts to reconstructed dense 3D trajectory stream. Our system and method are the first in reconstructing full body motion of more than five people engaged in social interactions without using markers. We also empirically demonstrate the impact of the number of views in achieving this goal.

This paper is aimed at calibrating the relative posture and position, i.e. extrinsic parameters, of a stationary camera against a 3D reference object which is not directly visible from the camera. We capture the reference object via a mirror under three different unknown poses, and then calibrate the extrinsic parameters from 2D appearances of reflections of the reference object in the mirrors.The key contribution of this paper is to present a new algorithm which returns a unique solution of three P3P problems from three mirrored images. While each P3P problem has up to four solutions and therefore a set of three P3P problems has up to 64 solutions, our method can select a solution based on an orthogonality constraint which should be satisfied by all families of reflections of a single reference object. In addition we propose a new scheme to compute the extrinsic parameters by solving a large system of linear equations. These two points enable us to provide a unique and robust solution.We demonstrate the advantages of the proposed method against a state-of-the-art by qualitative and quantitative evaluations using synthesized and real data.

No abstract