Robust Higher Order Potentials for Enforcing Label Consistency

Abstract

This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner. Our method is based on higher order conditional random fields and uses potentials defined on sets of pixels (image segments) generated using unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and can be seen as a generalization of the commonly used pairwise contrast sensitive smoothness potentials. The higher order potential functions used in our framework take the form of the Robust P n model and are more general than the P n Potts model recently proposed by Kohli et al. We prove that the optimal swap and expansion moves for energy functions composed of these potentials can be computed by solving a stmincut problem. This enables the use of powerful graph cut based move making algorithms for performing inference in the framework. We test our method on the problem of multi-class object segmentation by augmenting the conventional CRF used for object segmentation with higher order potentials defined on image regions. Experiments on challenging data sets show that integration of higher order potentials quantitatively and qualitatively improves results leading to much better definition of object boundaries. We P. Kohli ( )

Higher-order clique based image segmentation using evolutionary game theory

“…For comparative evaluation of our method, pair-wise graph cuts [1] and graph cuts for higher-order potentials defined on superpixels [9] are implemented. We test our algorithm on different datasets.…”

“…We compare our method with both pairwise graph cuts [1] and higher-order graph cuts. [9] While graph cuts penalizes neighboring pixels of taking different labels, the higher-order graph cuts would further enforce pixels within superpixel take the same label. We also compare our method with interactive higher-order segmentation, [38] where the higher-order formulation imposed the soft label consistency constraint on pixels within superpixels.…”

“…Higher-order clique potentials have the capability to model complex interactions of random variables. Compared with the pairwise model, experiments [7][8][9][10][11][12][13][14] showed superior results by introducing higher-order cliques, making it essential to find an efficient algorithm to solve higher-order energies. Although many methods have been proposed, the energy forms are simple and specified, which is far behind the need of effectively describing the underlying problem.…”

“…Although many methods have been proposed, the energy forms are simple and specified, which is far behind the need of effectively describing the underlying problem. Many of existing methods simply added a specified clique term to the pairwise energy, and solved the higher-order energies using either moving making algorithms, [7][8][9]13] or belief propagation, [11] or message passing. [10,12] Such inference algorithms are scale exponentially with the size of the maximal clique in the graph.…”

Detecting, segmenting and tracking unknown objects using multi-label MRF inference

“…(5) represents the first order cliques of the MRF, whereas the second half represents the second order cliques. Like many others we apply a contrast sensitive Potts model [49,50,2,51] for the inter-label dependencies f (l i , l j ). The EM algorithm used to find the estimateθ in Eq.…”