Estimating Human Pose with Flowing Puppets

Abstract

forward ow backward ow AbstractWe address the problem of upper-body human pose estimation in uncontrolled monocular video sequences, without manual initialization. Most current methods focus on isolated video frames and often fail to correctly localize arms and hands. Inferring pose over a video sequence is advantageous because poses of people in adjacent frames exhibit properties of smooth variation due to the nature of human and camera motion. To exploit this, previous methods have used prior knowledge about distinctive actions or generic temporal priors combined with static image likelihoods to track people in motion. Here we take a different approach based on a simple observation: Information about how a person moves from frame to frame is present in the optical flow field. We develop an approach for tracking articulated motions that "links" articulated shape models of people in adjacent frames through the dense optical flow. Key to this approach is a 2D shape model of the body that we use to compute how the body moves over time. The resulting "flowing puppets" provide a way of integrating image evidence across frames to improve pose inference. We apply our method on a challenging dataset of TV video sequences and show state-of-the-art performance.

Mixing Body-Part Sequences for Human Pose Estimation

“…Elbows Wrists Ours (ESIP+Recomb) 54.2 64.6 Tokola et al [28] 49.0 37.0 Zuffi et al [35] 52.0 42.0 Table 1, which are shown on the version of VideoPose dataset used in [12], and has one sequence less than the version used in [28] and [35].…”

“…More recently, Zuffi et al [35] have proposed a scheme where poses across two consecutive frames are coupled using optical flow. Although this method showed promising results, it is limited to frame-to-frame refinements.…”

PoseTrack: Joint Multi-person Pose Estimation and Tracking

“…Single person pose estimation in videos has also been studied extensively in the literature [28,9,46,33,46,20,44,29,13,18]. These approaches mainly aim to improve pose estimation by utilizing temporal smoothing constraints [28,9,44,33,13] and/or optical flow information [46,20,29], but they are not directly applicable to videos with multiple potentially occluding persons.…”

“…These approaches mainly aim to improve pose estimation by utilizing temporal smoothing constraints [28,9,44,33,13] and/or optical flow information [46,20,29], but they are not directly applicable to videos with multiple potentially occluding persons.…”

“…Many applications, such as mentioned before, however, aim to analyze human body motion over time. While there exists a notable number of works that track the pose of a single person in a video [28,9,44,33,46,20,29,7,13,18], multi-person human pose estimation in unconstrained videos has not been addressed in the literature. In this work, we address the problem of tracking the poses of multiple persons in an unconstrained setting.…”