Learning to Combine Bottom-Up and Top-Down Segmentation

Levin, Anat; Weiss, Yair

doi:10.1007/s11263-008-0166-0

Cited by 97 publications

(46 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The last few years have seen the emergence of object segmentation algorithms which integrate object specific top-down information with image based low-level features (Borenstein and Malik 2006;He et al 2004;Huang et al 2004;Kumar et al 2005;Levin and Weiss 2006). These methods have produced excellent results on challenging data sets.…”

Section: Object Segmentation and Recognitionmentioning

confidence: 99%

Robust Higher Order Potentials for Enforcing Label Consistency

2009

View full text Add to dashboard Cite

This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner. Our method is based on higher order conditional random fields and uses potentials defined on sets of pixels (image segments) generated using unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and can be seen as a generalization of the commonly used pairwise contrast sensitive smoothness potentials. The higher order potential functions used in our framework take the form of the Robust P n model and are more general than the P n Potts model recently proposed by Kohli et al. We prove that the optimal swap and expansion moves for energy functions composed of these potentials can be computed by solving a stmincut problem. This enables the use of powerful graph cut based move making algorithms for performing inference in the framework. We test our method on the problem of multi-class object segmentation by augmenting the conventional CRF used for object segmentation with higher order potentials defined on image regions. Experiments on challenging data sets show that integration of higher order potentials quantitatively and qualitatively improves results leading to much better definition of object boundaries. We P. Kohli ( )

show abstract

Section: Object Segmentation and Recognitionmentioning

confidence: 99%

Robust Higher Order Potentials for Enforcing Label Consistency

2009

View full text Add to dashboard Cite

show abstract

“…The horse dataset consists of side-view images of horses and was investigated in Borenstein et al (2004), Shotton et al (2005), Levin and Weiss (2006), Winn and Jojic (2005), Kumar et al (2005). While Borenstein et al (2004), Levin and Weiss (2006), Winn and Jojic (2005), Kumar et al (2005) concentrate their efforts mostly on segmentation of the detected objects, Shotton et al (2005) achieves impressive detection results using contour-based learning. Both datasets have a very high within-class variation in shape, color, and texture and present challenging classes of objects for testing our approach.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…The problem of class specific segmentations was addressed in several papers (Borenstein et al 2004;Kumar et al 2005;Winn and Jojic 2005;Levin and Weiss 2006;Wang et al 2007;Borenstein 2006;Leibe et al 2008), in many cases independent of detection. These studies further demonstrate the potential benefits of combining bottom-up with top-down information.…”

Section: Related Workmentioning

confidence: 99%

Shape Based Detection and Top-Down Delineation Using Image Segments

Gorelick

2009

Int J Comput Vis

View full text Add to dashboard Cite

We introduce a segmentation-based detection and top-down figure-ground delineation algorithm. Unlike common methods which use appearance for detection, our method relies primarily on the shape of objects as is reflected by their bottom-up segmentation.Our algorithm receives as input an image, along with its bottom-up hierarchical segmentation. The shape of each segment is then described both by its significant boundary sections and by regional, dense orientation information derived from the segment's shape using the Poisson equation. Our method then examines multiple, overlapping segmentation hypotheses, using their shape and color, in an attempt to find a "coherent whole," i.e., a collection of segments that consistently vote for an object at a single location in the image. Once an object is detected, we propose a novel pixellevel top-down figure-ground segmentation by "competitive coverage" process to accurately delineate the boundaries of the object. In this process, given a particular detection hypothesis, we let the voting segments compete for interpreting (covering) each of the semantic parts of an object. Incorporating competition in the process allows us to resolve ambiguities that arise when two different regions are matched to the same object part and to discard nearby false regions that participated in the voting process.We provide quantitative and qualitative experimental results on challenging datasets. These experiments demonstrate that our method can accurately detect and segment objects with complex shapes, obtaining results comparable to those of existing state of the art methods. Moreover, our method allows us to simultaneously detect multiple instances of class objects in images and to cope with challenging types of occlusions such as occlusions by a bar of varying size or by another object of the same class, that are difficult to handle with other existing class-specific top-down segmentation methods.

show abstract

“…It is usually used by top-down object segmentation approaches, which use the mid-level context scale to disambiguate local predictions and, in contrast to bottom-up approaches, they use a priori knowledge about the whole object such as its structure (Levin and Weiss 2009). They incorporate global object properties, like shape masks or histograms of oriented gradients (Yang et al 2010;Leibe et al 2008;Winn and Jojic 2005;Lempitsky et al 2009;Carreira and Sminchisescu 2010).…”

Section: Mid-level Scalementioning

confidence: 99%

Harmony Potentials

et al. 2011

View full text Add to dashboard Cite

The Hierarchical Conditional Random Field (HCRF) model have been successfully applied to a number of image labeling problems, including image segmentation. However, existing HCRF models of image segmentation do not allow multiple classes to be assigned to a single region, which limits their ability to incorporate contextual information across multiple scales. At higher scales in the image, this representation yields an oversimplified model since multiple classes can be reasonably expected to appear within large regions. This simplified model particularly limits the impact of information at higher scales. Since classlabel information at these scales is usually more reliable than at lower, noisier scales, neglecting this information is undesirable. To address these issues, we propose a new consistency potential for image labeling problems, which we call the harmony potential. It can encode any possible combination of labels, penalizing only unlikely combinations of classes. We also propose an effective sampling strategy over this expanded label set that renders tractable the underlying optimization problem. Our approach obtains state-of-theart results on two challenging, standard benchmark datasets Both authors contributed equally to this work.

show abstract

Learning to Combine Bottom-Up and Top-Down Segmentation

Cited by 97 publications

References 15 publications

Robust Higher Order Potentials for Enforcing Label Consistency

Robust Higher Order Potentials for Enforcing Label Consistency

Shape Based Detection and Top-Down Delineation Using Image Segments

Harmony Potentials

Contact Info

Product

Resources

About