Joint Recovery of Dense Correspondence and Cosegmentation in Two Images

Taniai, Tatsunori; Sinha, Sudipta N.; Sato, Yoichi

doi:10.1109/cvpr.2016.460

Cited by 121 publications

(252 citation statements)

References 45 publications

Supporting

Mentioning

249

Contrasting

Order By: Relevance

“…Furthermore, since limited training data is available for semantic correspondence, we propose a weakly-supervised feature learning scheme that leverages correspondence consistency between object locations provided in existing image datasets. Experimental results show that the FCSS descriptor outperforms conventional handcrafted descriptors and CNN-based descriptors on various benchmarks, including that of Taniai et al [45], Proposal Flow [18], the PASCAL dataset [6], and Caltech-101 [13].…”

Section: Introductionmentioning

confidence: 97%

“…Avg. DFF [49] 0.495 0.304 0.224 0.341 DSP [24] 0.487 0.465 0.382 0.445 SIFT Flow [31] 0.632 0.509 0.360 0.500 Zhou et al [53] 0.721 0.514 0.436 0.556 Taniai et al [45] 0.830 0.595 0.483 0.636 Proposal Flow [18] 0.786 0.653 0.531 0.657 FCSS w/DSP [24] 0.527 0.580 0.439 0.515 FCSS w/SF [31] 0.830 0.656 0.494 0.660 FCSS w/PF [18] 0.839 0.635 0.582 0.685 Table 2. Matching accuracy compared to state-of-the-art correspondence techniques on the Taniai benchmark [45].…”

Section: Experimental Settingsmentioning

confidence: 99%

“…[19] 0.561 0.380 0.270 0.404 LIFT [50] 0.730 0.318 0.306 0.451 VGG [42] 0 Table 1. Matching accuracy for various feature descriptors with fixed SF optimization on the Taniai benchmark [45]. VGG w/S-CSS † denotes results with randomly selected sampling patterns.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…Among these methods are some based on SIFT Flow [31,24], which uses hierarchical dual-layer belief propagation (BP). Other instances include the methods with an exemplar-LDA approach [2], through joint image set alignment [54], or together with cosegmentation [45].…”

Section: Related Workmentioning

confidence: 99%

“…These variations, in addition to other complications from occlusion and background clutter, lead to significant differences in appearance that can distract matching by handcrafted feature descriptors [34,46]. Although powerful optimization techniques can help by enforcing smoothness constraints over a correspondence map [31,24,54,45,18], Figure 1. Visualization of local self-similarity.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence

Kim

Min

Ham

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

We present a descriptor, called fully convolutional selfsimilarity (FCSS), for dense semantic correspondence. To robustly match points among different instances within the same object class, we formulate FCSS using local selfsimilarity (LSS) within a fully convolutional network. In contrast to existing CNN-based descriptors, FCSS is inherently insensitive to intra-class appearance variations because of its LSS-based structure, while maintaining the precise localization ability of deep neural networks. The sampling patterns of local structure and the self-similarity measure are jointly learned within the proposed network in an end-to-end and multi-scale manner. As training data for semantic correspondence is rather limited, we propose to leverage object candidate priors provided in existing image datasets and also correspondence consistency between object pairs to enable weakly-supervised learning. Experiments demonstrate that FCSS outperforms conventional handcrafted descriptors and CNN-based descriptors on various benchmarks.

show abstract

Section: Introductionmentioning

confidence: 97%

Section: Experimental Settingsmentioning

confidence: 99%

Section: Experimental Settingsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence

Kim

Min

Ham

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

Attentive Semantic Alignment with Offset-Aware Correlation Kernels

Seo

Lee

Jung

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Semantic correspondence is the problem of establishing correspondences across images depicting different instances of the same object or scene class. One of recent approaches to this problem is to estimate parameters of a global transformation model that densely aligns one image to the other. Since an entire correlation map between all feature pairs across images is typically used to predict such a global transformation, noisy features from different backgrounds, clutter, and occlusion distract the predictor from correct estimation of the alignment. This is a challenging issue, in particular, in the problem of semantic correspondence where a large degree of image variations is often involved. In this paper, we introduce an attentive semantic alignment method that focuses on reliable correlations, filtering out distractors. For effective attention, we also propose an offset-aware correlation kernel that learns to capture translation-invariant local transformations in computing correlation values over spatial locations. Experiments demonstrate the effectiveness of the attentive model and offset-aware kernel, and the proposed model combining both techniques achieves the state-of-the-art performance.

show abstract