Space-Time Body Pose Estimation in Uncontrolled Environments

Abstract

We propose a data-driven, multi-view body pose estimation algorithm for video. It can operate in uncontrolled environments with loosely calibrated and low resolution cameras and without restricting assumptions on the family of possible poses or motions.Our algorithm first estimates a rough pose estimation using a spatial and temporal silhouette based search in a database of known poses. The estimated pose is improved in a novel pose consistency step acting locally on single frames and globally over the entire sequence. Finally, the resulting pose estimation is refined in a spatial and temporal pose optimization consisting of novel constraints to obtain an accurate pose. Our method proved to perform well on low resolution video footage from real broadcast of soccer games.

Lower Body Pose Estimation in Team Sports Videos Using Label-Grid Classifier Integrated with Tracking-by-Detection

“…Our framework can be viewed as a revisited version of Ref. [16] by using machinelearning and dense visual features. In other words, traditional silhouette matching strategy for pose estimation was innovated by our approach using HOG features and Random Decision Forests to embed all pose appearance patterns into the randomized feature space.…”

“…Our HOG-based classification approach can be viewed as the modern replacement of the classical silhouette-matching schemes using background subtraction, such as Ref. [16]. We instead use randomized HOG features (learned by Random Decision Forests) to robustly classify the pose with machine learning.…”

“…Germann et al [16] proposed silhouette-based database matching methods for soccer players. These methods first construct an exemplar-pose database with silhouette images of players.…”

Synchronization of user-generated videos through trajectory correspondence and a refinement procedure

“…The work of [14] has tried to extend the self-similarity matrices for characterizing group activities; they assume very precise knowledge of 2D trajectories of humans/objects. The work of [9] estimates poses of the players in sports videos for aiding the task of video synchronization. However, their method relies on precise silhouette extraction of the various people involved in the video and thus is very limited.…”

“…Some researchers [9] have tried to estimate poses of subjects in sports events like hockey; however, their methods are far from automatic and rely heavily on manual extraction of human silhouettes for all frames of the video. We thus resort to activity descriptor based methods.…”