Are spatial and global constraints really necessary for segmentation?

Abstract

Many state-of-the-art segmentation algorithms rely on Markov or Conditional Random Field models designed to enforce spatial and global consistency constraints. This is often accomplished by introducing additional latent variables to the model, which can greatly increase its complexity. As a result, estimating the model parameters or computing the best maximum a posteriori (MAP) assignment becomes a computationally expensive task.In a series of experiments on the PASCAL and the MSRC datasets, we were unable to find evidence of a significant performance increase attributed to the introduction of such constraints. On the contrary, we found that similar levels of performance can be achieved using a much simpler design that essentially ignores these constraints. This more simple approach makes use of the same local and global features to leverage evidence from the image, but instead directly biases the preferences of individual pixels. While our investigation does not prove that spatial and consistency constraints are not useful in principle, it points to the conclusion that they should be validated in a larger context.

Shortest-Path Constraints for 3D Multiobject Semiautomatic Segmentation Via Clustering and Graph Cut

“…Pairwise priors learned from training sets might not satisfy this property. Therefore some authors truncate nonsubmodular terms in order to optimize with Graph Cut algorithms [24], and others resort to arbitrary energy minimizers [18].…”

“…The cluster graph is then partitioned based on color information yielding a coarse segmentation of the image. Recent conditional random field-based (CRF) segmentation approaches such as [17], [18] rely on image clustering not only to reduce memory overhead but also to collect image features from clusters and their neighborhood.…”

“…It has appeared in scene layout estimation and segmentation where spatial relationships are predefined on a limited number of classes [23], and in part-based object detection and segmentation where such relationships are learned from annotated training data [18], [24]. [25] segment multiple or compound objects encountered in relative positions of containment and exclusion, also introducing constraints on preferred distances between the boundaries of regions.…”

Structured Image Segmentation Using Kernelized Features

“…Although the above definition depends only on the labels y i and y j , the pairwise term can, in fact, be made data-aware (as in [2,7]). For instance, it can be made gradient-adaptive by including parameters for each discretized gradient level.…”

“…This is because they are more objective-driven and do not involve the daunting partition function that can render maximum-likelihood approaches intractable in CRFs with loopy graph structures. Compared with earlier approaches including the max-margin Markov network [4], the structured support vector machine (SSVM) [1] is especially appealing, and has since been successfully applied to many computer vision tasks, such as in [5][6][7], among others. The SSVM's appeal is due, in part, to its ability to take into account a variety of loss functions.…”