Adaptive pose priors for pictorial structures

Abstract

Pictorial structure (PS) models are extensively used for part-based recognition of scenes, people, animals and multi-part objects. To achieve tractability, the structure and parameterization of the model is often restricted, for example, by assuming tree dependency structure and unimodal, data-independent pairwise interactions. These expressivity restrictions fail to capture important patterns in the data. On the other hand, local methods such as nearest-neighbor classification and kernel density estimation provide nonparametric flexibility but require large amounts of data to generalize well. We propose a simple semi-parametric approach that combines the tractability of pictorial structure inference with the flexibility of non-parametric methods by expressing a subset of model parameters as kernel regression estimates from a learned sparse set of exemplars. This yields query-specific, image-dependent pose priors. We develop an effective shape-based kernel for upper-body pose similarity and propose a leave-one-out loss function for learning a sparse subset of exemplars for kernel regression. We apply our techniques to two challenging datasets of human figure parsing and advance the state-of-the-art (from 80% to 86% on the Buffy dataset [8]), while using only 15% of the training data as exemplars. DisciplinesComputer Sciences AbstractPictorial structure (PS) models are extensively used for part-based recognition of scenes, people, animals and multi-part objects. To achieve tractability, the structure and parameterization of the model is often restricted, for example, by assuming tree dependency structure and unimodal, data-independent pairwise interactions. These expressivity restrictions fail to capture important patterns in the data. On the other hand, local methods such as nearest-neighbor classification and kernel density estimation provide nonparametric flexibility but require large amounts of data to generalize well. We propose a simple semi-parametric approach that combines the tractability of pictorial structure inference with the flexibility of non-parametric methods by expressing a subset of model parameters as kernel regression estimates from a learned sparse set of exemplars. This yields query-specific, image-dependent pose priors. We develop an effective shape-based kernel for upper-body pose similarity and propose a leave-one-out loss function for learning a sparse subset of exemplars for kernel regression. We apply our techniques to two challenging datasets of human figure parsing and advance the state-of-the-art (from 80% to 86% on the Buffy dataset [8]), while using only 15% of the training data as exemplars.

Active Learning for Human Pose Estimation

“…Pictorial structures are one of the classical approaches to articulated pose estimation [2,12,14,40,33]. In these methods, spatial correlations between parts of the body are expressed as a tree-structured graphical model with kinematic priors that couple connected limbs.…”

Poselet Conditioned Pictorial Structures

“…In pose estimation literature the method of [19] is probably the closest to ours. Similar to their work, we define a PS model where unary and pairwise terms are image conditioned.…”

“…In contrast they rely on silhouette based similarity cues that are ineffective in the presence of background clutter, and act relatively local and thus capture mostly local pairwise part interactions. This makes our method applicable to more challenging sport images showing highly articulated humans from different viewpoints, while the method of [19] has been applied to frontal poses only with a comparatively small degree of articulation.…”

Modeling Motion of Body Parts for Action Recognition

“…The use of these models has been demonstrated for robust detection of object parts as well as for part-based recognition of articulated objects [3,11,27]. In the context of human action recognition, it is reasonable to assume that each action can be described as a combination of movements of different body parts (e.g, head, hands, legs and feet).…”

“…The proposed representation combines the advantages of both local and global representations, encoding the relevant motion information as well as being robust to local appearance changes. Our representation is motivated by the recent success of pictorial structures [3,11,27] and our recognition framework is inspired by the fundamentals of sparse representation for recognition [32]. The contributions of our work are:…”