We propose to bridge the gap between Random Field (RF) formulations for joint categorization and segmentation (JCaS), which model local interactions among pixels and superpixels, and Bag of Features categorization algorithms, which use global descriptors. For this purpose, we introduce new higher order potentials that encode the classification cost of a histogram extracted from all the objects in an image that belong to a particular category, where the cost is given as the output of a classifier when applied to the histogram. The potentials efficiently encode the classification costs of several histograms resulting from the different possible segmentations of an image. They can be integrated with existing potentials, hence providing a natural unification of global and local interactions. The potentials' parameters can be treated as parameters of the RF and hence be jointly learnt along with the other parameters of the RF. Experiments show that our framework can be used to improve the performance of existing JCaS algorithms.

Abstract

Interactive image segmentation traditionally involves the use of algorithms such as Graph Cuts or Random Walker. Common concerns with using Graph Cuts are metrication artifacts (blockiness) and the shrinking bias (bias towards shorter boundaries). The Random Walker avoids these problems, but suffers from the proximity bias (sensitivity to location of pixels labeled by the user). In this work, we introduce a new family of segmentation algorithms that includes Graph Cuts and Random Walker as special cases. We explore image segmentation using continuous-valued Markov Random Fields (MRFs) with probability distributions following the p-norm of the difference between configurations of neighboring sites. For p=1 these MRFs may be interpreted as the standard binary MRF used by Graph Cuts, while for p=2 these MRFs may be viewed as Gaussian MRFs employed by the Random Walker algorithm. By allowing the probability distribution for neighboring sites to take any arbitrary p-norm (p ≥ 1), we pave the path for hybrid extensions of these algorithms. Experiments show that the use of a fractional p (1 < p < 2) can be used to resolve the aforementioned drawbacks of these algorithms.

We present a closed form solution to the problem of segmenting multiple 2-D motion models of the same type directly from the partial derivatives of an image sequence. We introduce the multibody brightness constancy constraint (MBCC), a polynomial equation relating motion models, image derivatives and pixel coordinates that is independent of the segmentation of the image measurements. We first show that the optical flow at a pixel can be obtained analytically as the derivative of the MBCC at the corresponding image measurement, without knowing the motion model associated with that pixel. We then show that the parameters of the multiple motion models can be obtained from the cross products of the derivatives of the MBCC at a set of image measurements that minimize a suitable distance function. Our approach requires no feature tracking, point correspondences or optical flow, and provides a global noniterative solution that can be used to initialize more expensive iterative approaches to motion segmentation. Experiments on real and synthetic sequences are also presented.

No abstract