Nonlinearly Constrained MRFs: Exploring the Intrinsic Dimensions of Higher-Order Cliques

Abstract

This paper introduces an efficient approach to integrating non-local statistics into the higher-order Markov Random Fields (MRFs) framework. Motivated by the observation that many non-local statistics (e.g., shape priors, color distributions) can usually be represented by a small number of parameters, we reformulate the higher-order MRF model by introducing additional latent variables to represent the intrinsic dimensions of the higher-order cliques. The resulting new model, called NC-MRF, not only provides the flexibility in representing the configurations of higher-order cliques, but also automatically decomposes the energy function into less coupled terms, allowing us to design an efficient algorithmic framework for maximum a posteriori (MAP) inference. Based on this novel modeling/inference framework, we achieve state-of-the-art solutions to the challenging problems of class-specific image segmentation and template-based 3D facial expression tracking, which demonstrate the potential of our approach.

Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey

“…This approach has been shown to achieve much better efficiency than applying those standard algorithms to address the original problem directly. Very recently, [228] took a further step along this line of research by exploring the intrinsic dimensions of higher-order cliques, and proposed a powerful MRF-based modeling/inference framework (called NC-MRF) which signi cantly broadens the applicability of higher-order MRFs in visual perception.…”

Higher-order clique based image segmentation using evolutionary game theory

“…1 only describes constraints between pixel pairs. In order to capture rich statistics of the image, Zeng et al [14] introduced a framework to integrate non-local statistics into the higher-order Markov Random Fields, using additional latent variables to represent the intrinsic dimensions of the higher-order cliques. Jain et al [35] solved the higher-order clustering problem by combining attributes of both decomposition of higher-order similarity measures for use in spectral clustering and explicitly use low-rank matrix representations.…”

“…Higher-order clique potentials have the capability to model complex interactions of random variables. Compared with the pairwise model, experiments [7][8][9][10][11][12][13][14] showed superior results by introducing higher-order cliques, making it essential to find an efficient algorithm to solve higher-order energies. Although many methods have been proposed, the energy forms are simple and specified, which is far behind the need of effectively describing the underlying problem.…”