Real-Time Tracking of Articulated Human Models Using a 3D Shape-from-Silhouette Method

Abstract

This paper describes a system which acquires 3D data and uses the data to track an eleven degree of freedom human model in real-time. Using four cameras we create a time-varying volumetric image (a visual hull) of whatever object is moving in the space observed by all four cameras. We are currently operating the sensor in a volume of approximately 500,000 voxels (1.5 inch cubes) at a rate of 25 Hz. We are able to track the upper body dynamics of a human within the workspace. The system is able to track the x, y position of the body, a torso rotation about the z axis, along with four rotations in each arm. Tracking occurs using a method we developed to be extremely fas~which allows both data acquisition and tracking to occur on one computer at a rate of 16 Hz. We also developed a calibration procedure, which allows the system to be reconfigured or even moved and to quickly be recalibrated. Furthermore the system utilizes another computer to visualize either the voxel data overlaid with the joint locations or to view a human avatar, both of which are driven in real-time. Lastly our system has been implemented to perform crane gesture recognition, and has been linked with a large robotic arm to simulate crane movements. Corresponding author

Real-Time 3D Body Pose Estimation

“…Voxels lying outside of the silhouette in at least one view do not belong to the intersection of the cones and can be discarded. Because of their simplicity, voxel-based procedures have been used for body tracking [26][27][28][29][30]. Their drawback is their tendency to be expensive as a high number must be projected into the image planes.…”

Development of Advanced Active Haptic System for Musculokelelton-Exoskeleton Interactions

“…In the computer vision community, researchers have built kinematic models of the human body in order to determine a sequence of poses of the body in video succession [1][2][3]. These models are not very useful in our application, however, since they do not model or predict motions.…”

“…The tasks involve single and double arm movements, gripping, single and double arm lifting, normal walking, stair walking, and stepping over objects. 3 Second Year Research:…”

Image-based reconstruction for view-independent human motion recognition

“…In [11], Luck et. al., generate a 3D voxelized model of a moving human, extracting the moving form using adaptive background subtraction and thresholding in real-time.…”