Finding Deformable Shapes Using Loopy Belief Propagation

Abstract

A novel deformable template is presented which detects and localizes shapes in grayscale images. The template is formulated as a Bayesian graphical model of a two-dimensional shape contour, and it is matched to the image using a variant of the belief propagation (BP) algorithm used for inference on graphical models. The algorithm can localize a target shape contour in a cluttered image and can accommodate arbitrary global translation and rotation of the target as well as significant shape deformations, without requiring the template to be initialized in any special way (e.g. near the target).The use of BP removes a serious restriction imposed in related earlier work, in which the matching is performed by dynamic programming and thus requires the graphical model to be tree-shaped (i.e. without loops). Although BP is not guaranteed to converge when applied to inference on non-tree-shaped graphs, we find empirically that it does converge even for deformable template models with one or more loops. To speed up the BP algorithm, we augment it by a pruning procedure and a novel technique, inspired by the 20 Questions (divide-and-conquer) search strategy, called "focused message updating." These modifications boost the speed of convergence by over an order of magnitude, resulting in an algorithm that detects and localizes shapes in grayscale images in as little as several seconds on an 850 MHz AMD processor.

Exemplar-Based Graph Matching for Robust Facial Landmark Localization

“…Among a number of non-parametric shape priors, Markov random field (MRF) [8] is perhaps the most natural way to govern the geometrical configuration of a point set. For instance, Liang et al [23] proposed a constrained MRF by regularizing the shape with a PCA-based prior.…”

Max Margin Learning of Hierarchical Configural Deformable Templates (HCDTs) for Efficient Object Parsing and Pose Estimation

“…Most computer vision systems only address one of these tasks. There has been influential work on weakdetection (Coughlan and Ferreira 2002) and on the related problem of registration (Chui and Rangarajan 2000;Belongie et al 2002). Work on segmentation includes (Kumar et al 2005;Leibe et al 2004;Borenstein and Malik 2006;Cour and Shi 2007;Levin and Weiss 2006;Winn andJojic 2005 and.…”

“…But although DP is guaranteed to be polynomial in the relevant quantities (number of layers and graph nodes, the size of state space of the node variables z and t) full DP is too slow because of the large size of the state space-every sub-part of the object can occur in any position of the image, at any orientation, and any scale (recall that z ν = (x ν , θ ν , s ν )) so the states of the nodes depend on the image size, the number of allowable orientations, and the allowable range of sizes of the sub-parts). Hence, as in other applications of DP or BP to vision (Coughlan et al 2000;Coughlan and Ferreira Input: Fig. 6 The inference algorithm.…”

“…These proposals are selected by two mechanisms: (i) energy pruning-to remove proposals corresponding to large energy, and (ii) surround suppression-to suppress proposals that occur within a surrounding window in z (similar to non-maximal suppression). Intuitively, the first pruning mechanism rejects high energy configurations (which is a standard pruning technique for DP-see Coughlan et al 2000;Coughlan and Ferreira 2002) while the second mechanism rejects states which are too similar. The second pruning mechanism is less commonly used but the errors it may cause are fairly benign-e.g.…”

Rotation Invariant Non-rigid Shape Matching in Cluttered Scenes

“…In [21], it was extended to match regions of a shape. Belief propagation was used in [22] to match shapes where shapes with loops or holes are allowed.…”