Cosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs

Rother, Carsten; Minka, Tom; Blake, A.; Kolmogorov, Vladimir

doi:10.1109/cvpr.2006.91

Cited by 429 publications

(574 citation statements)

References 18 publications

Supporting

Mentioning

572

Contrasting

Unclassified

Order By: Relevance

“…Our interest is to establish dense correspondences between images across scenes, an alignment problem that can be more challenging than aligning images from the same scene and aligning images of the same object category since we wish all the elements that compose the scene to be aligned. Our work relates to the task of co-segmentation [41] that tried to simultaneously segment the common parts of an image pair, and to the problem of shape matching [5] that was used in the context of object recognition.…”

Section: Related Workmentioning

confidence: 99%

SIFT Flow: Dense Correspondence Across Scenes and Its Applications

Liu¹,

Yuen²,

Torralba³

2016

Dense Image Correspondences for Computer Vision

458

855

View full text Add to dashboard Cite

Abstract-While image alignment has been studied in different areas of computer vision for decades, aligning images depicting different scenes remains a challenging problem. Analogous to optical flow where an image is aligned to its temporally adjacent frame, we propose SIFT flow, a method to align an image to its nearest neighbors in a large image corpus containing a variety of scenes. The SIFT flow algorithm consists of matching densely sampled, pixel-wise SIFT features between two images, while preserving spatial discontinuities. The SIFT features allow robust matching across different scene/object appearances, whereas the discontinuitypreserving spatial model allows matching of objects located at different parts of the scene. Experiments show that the proposed approach robustly aligns complex scene pairs containing significant spatial differences. Based on SIFT flow, we propose an alignmentbased large database framework for image analysis and synthesis, where image information is transferred from the nearest neighbors to a query image according to the dense scene correspondence. This framework is demonstrated through concrete applications, such as motion field prediction from a single image, motion synthesis via object transfer, satellite image registration and face recognition.

show abstract

Section: Related Workmentioning

confidence: 99%

SIFT Flow: Dense Correspondence Across Scenes and Its Applications

Liu¹,

Yuen²,

Torralba³

2016

Dense Image Correspondences for Computer Vision

458

855

View full text Add to dashboard Cite

show abstract

“…Co-segmentation was first introduced by Rother et al (2006), who used histogram matching to simultaneously segment the same object in two different images. Since then, numerous methods were proposed to improve and refine the co-segmentation (Mukherjee et al 2009;Hochbaum and Singh 2009;Batra et al 2010;Joulin et al 2010), many of which work in the context of a pair of images with the exact same object Mukherjee et al 2009;Hochbaum and Singh 2009) or require some form of user interaction (Batra et al 2010;Collins et al 2012).…”

Section: Object Discovery and Segmentationmentioning

confidence: 99%

Joint Inference in Weakly-Annotated Image Datasets via Dense Correspondence

Rubinstein

Liu

Freeman

2016

Dense Image Correspondences for Computer Vision

View full text Add to dashboard Cite

We present a principled framework for inferring pixel labels in weakly-annotated image datasets. Most previous, example-based approaches to computer vision rely on a large corpus of densely labeled images. However, for large, modern image datasets, such labels are expensive to obtain and are often unavailable. We establish a large-scale graphical model spanning all labeled and unlabeled images, then solve it to infer pixel labels jointly for all images in the dataset while enforcing consistent annotations over similar visual patterns. This model requires significantly less labeled data and assists in resolving ambiguities by propagating inferred annotations from images with stronger local visual evidences to images with weaker local evidences. We apply our proposed framework to two computer vision problems, namely image annotation with semantic segmentation, and object discovery and co-segmentation (segmenting multiple images containing a common object). Extensive numerical evaluations and comparisons show that our method consistently outperforms the state-of-the-art in automatic annotation and semantic labeling, while requiring significantly less labeled data. In contrast to previous co-segmentation techniques, our method manages to discover and segment objects well even in the presence of substantial amounts of noise images (images not containing the common object), as typical for datasets collected from Internet search.

show abstract

“…Because the inputs are known to share some visual relationship-for example, they contain the same foreground object, or instances of the same object class-the algorithm has valuable cues about which pixels might go together. At a high level, the idea is to detect any common appearance/shapes, exploit that association to determine likely foreground regions, then use a "shared" foreground model to jointly guide the region estimates in all input images [1][2][3][4][5][6][7][8]. In contrast, such cues are not available in the traditional single-image segmentation task, where the system must rely solely on bottom-up features to perform the grouping.…”

Section: Introductionmentioning

confidence: 99%

“…Researchers have made substantial progress on the cosegmentation problem in recent years. While initially the problem was defined to entail two input images showing very same object against distinct backgrounds [1], recent work broadens the problem definition to include batches of input images known only to contain instances of the same object class [2][3][4][5][6][7][8][9][10]. This is also referred to as weakly supervised or joint foreground segmentation: each input image is known to contain an instance from the same object category, but its localization within the background is unknown.…”

Section: Introductionmentioning

confidence: 99%

“…This is also referred to as weakly supervised or joint foreground segmentation: each input image is known to contain an instance from the same object category, but its localization within the background is unknown. 1 Some work further relaxes the two-region (foreground/background) assumption to tackle k-region segmentation [11,7,6]. Furthermore, eager to capitalize on large collections of weakly labeled images, methods are being developed to account for both noisily labeled instances [11,8] and scalable optimization [12,7,6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Which Image Pairs Will Cosegment Well? Predicting Partners for Cosegmentation

Jain

Grauman

2015

Computer Vision -- ACCV 2014

View full text Add to dashboard Cite

Abstract. Cosegmentation methods segment multiple related images jointly, exploiting their shared appearance to generate more robust foreground models. While existing approaches assume that an oracle will specify which pairs of images are amenable to cosegmentation, in many scenarios such external information may be difficult to obtain. This is problematic, since coupling the "wrong" images for segmentationeven images of the same object class-can actually deteriorate performance relative to single-image segmentation. Rather than manually specify partner images for cosegmentation, we propose to automatically predict which images will cosegment well together. We develop a learning-torank approach that identifies good partners, based on paired descriptors capturing the images' amenability to joint segmentation. We compare our approach to alternative methods for partnering images, including basic image similarity, and show the advantages on two challenging datasets.

show abstract

Cosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs

Cited by 429 publications

References 18 publications

SIFT Flow: Dense Correspondence Across Scenes and Its Applications

SIFT Flow: Dense Correspondence Across Scenes and Its Applications

Joint Inference in Weakly-Annotated Image Datasets via Dense Correspondence

Which Image Pairs Will Cosegment Well? Predicting Partners for Cosegmentation

Contact Info

Product

Resources

About