Harmony Potentials

Abstract

The Hierarchical Conditional Random Field (HCRF) model have been successfully applied to a number of image labeling problems, including image segmentation. However, existing HCRF models of image segmentation do not allow multiple classes to be assigned to a single region, which limits their ability to incorporate contextual information across multiple scales. At higher scales in the image, this representation yields an oversimplified model since multiple classes can be reasonably expected to appear within large regions. This simplified model particularly limits the impact of information at higher scales. Since classlabel information at these scales is usually more reliable than at lower, noisier scales, neglecting this information is undesirable. To address these issues, we propose a new consistency potential for image labeling problems, which we call the harmony potential. It can encode any possible combination of labels, penalizing only unlikely combinations of classes. We also propose an effective sampling strategy over this expanded label set that renders tractable the underlying optimization problem. Our approach obtains state-of-theart results on two challenging, standard benchmark datasets Both authors contributed equally to this work.

Weakly Supervised Graph Based Semantic Segmentation by Learning Communities of Image-Parts

“…Our approach achieves a higher average per-class accuracy than the baseline methods. We have also included the state of the art results of fully-supervised methods [7], [3], [36], and [50] reported on MSRC-21 dataset. Our approach is even comparable with the fully-supervised ones without requiring ground-truth labeling at training time.…”

“…A detailed analysis of computational complexity of our approach shall be discussed shortly. Table 1: Per-class accuracy on MSRC-21 using the proposed approach, state of the art fully supervised approaches ( [7], [3], [36], [50] ) and weakly supervised methods ( [44], [48], [53] Table 2: Performance accuracy on LabelMe dataset using the proposed approach, state of the art fully supervised approaches ( [41], [43], [31]) and weakly supervised methods ( [44], [30], [53]). The best results achieved by FS and WS approaches are highlighted in bold.…”

“…There is a wealth of published literature on fully supervised semantic segmentation that rely on location information associated with image labels (fully segmented training data) [3,5,7,8,9,17,18,20,23,25,26,29,31,36,40,41,42,45,49,50,55]. The aforementioned approaches require that every pixel in the training set be manually labeled.…”

Improved hierarchical conditional random field model for object segmentation

“…Multi-class object segmentation can be considered as a labeling problem that attempts to assign a label from a predefined label set to each pixel or superpixel in a given image, and group the pixels into one or more meaningful objects [1,2]. Many different labeling approaches have been developed and reported with some degree of success.…”

“…This is not sufficient to prohibit some unlikely classes appearing together. In [2], a harmony model as shown in Fig. 1d was proposed.…”