Estimating Human Dynamics On-the-fly Using Monocular Video For Pose Estimation

Abstract

Abstract-Human pose estimation using uncalibrated monocular visual inputs alone is a challenging problem for both the computer vision and robotics communities. From the robotics perspective, the challenge here is one of pose estimation of a multiply-articulated system of bodies using a single nonspecialized environmental sensor (the camera) and thereby, creating low-order surrogate computational models for analysis and control.In this work, we propose a technique for estimating the lowerlimb dynamics of a human solely based on captured behavior using an uncalibrated monocular video camera. We leverage our previously developed framework for human pose estimation to (i) deduce the correct sequence of temporally coherent gap-filled pose estimates, (ii) estimate physical parameters, employing a dynamics model incorporating the anthropometric constraints, and (iii) filter out the optimized gap-filled pose estimates, using an Unscented Kalman Filter (UKF) with the estimated dynamicallyequivalent human dynamics model. We test the framework on videos from the publicly available DARPA Mind's Eye Year 1 corpus [8]. The combined estimation and filtering framework not only results in more accurate physically plausible pose estimates, but also provides pose estimates for frames, where the original human pose estimation framework failed to provide one.

Watch-Bot: Unsupervised learning for reminding humans of forgotten actions

“…We assign object-topic to each object-word w o nd from P latent objecttopics, indicating which object-topic is interacted within the clip. The assignments are denoted as z (1) nd and z (2) nd . We use superscripts (1), (2) to denote action-topics and object-topics respectively.…”

“…We model the co-occurrence by drawing their priors from a mixture distribution. In the graphic model, π (1) kd , π (2) pd decide the probability of action-topic k and object-topic p occurring in a document d, where…”

“…There are a large number of works on vision-based human activity recognition for robots. These works infer the semantic label of the overall activity or localize actions in the complex activity for better human-robot interactions [20], [2], [10], assistive robotics [14], [34]. Given the input RGB/RGB-D videos [28], [17], [8], or 3D human joint motions [21], [26], or from other inertial/location sensors [9], [22], they train the perception model using fully or weekly labeled actions [17], [7], [13], or locations of annotated human/their interactive objects [30], [24].…”

Athlete Pose Estimation from Monocular TV Sports Footage

“…Model-based tracking is a derived technique which focuses on tracking the pose estimate from one time step to the next, starting from a known initialization based on an approximate dynamical model. Most generative pose estimation frameworks suffer from the fact that the optimisation is prone to hitting local minima, requiring a good initialisation and often failing on complex motions [2]. Pictorial Structures (PS) is a probabilistic inference for a tree-structured graphical model where the overall cost function for a pose decomposes across edges and nodes of the tree.…”

“…Most algorithms focus on recovering pose from single images and/or do not make full use of the temporal constraint on limb motion [3,17,14,2]. Available off-the-shelf single image methods such as [7] have trouble coping with the difficulty of the data, even when the human is detected.…”