Semantic Attribute Matching Networks

Abstract

We present semantic attribute matching networks (SAM-Net) for jointly establishing correspondences and transferring attributes across semantically similar images, which intelligently weaves the advantages of the two tasks while overcoming their limitations. SAM-Net accomplishes this through an iterative process of establishing reliable correspondences by reducing the attribute discrepancy between the images and synthesizing attribute transferred images using the learned correspondences. To learn the networks using weak supervisions in the form of image pairs, we present a semantic attribute matching loss based on the matching similarity between an attribute transferred source feature and a warped target feature. With SAM-Net, the state-ofthe-art performance is attained on several benchmarks for semantic matching and attribute transfer.

GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences

“…We thereby achieve a universal network for predicting dense flow fields, applicable to geometric matching, semantic correspondences and optical flow. The overall architecture follows a CNN feature-based coarse-to-fine strategy, which has proved widely successful for specific tasks [23,30,33,41,57]. However, contrary to previous works, our architecture combines global and local correlation layers, as discussed in Section 3.1 and 3.2, to benefit from their complementary properties.…”

“…We compare to several recent state-of-the-art methods specialised in semantic matching [22,30,31,33,48,49]. In addition to our universal network, we evaluate a version that adopts two architectural details that are used in the semantic correspondence literature.…”

“…Recently, Jeon et al [30] introduced PARN, a pyramidal model where dense affine transformation fields are progressively estimated in a coarse-tofine manner. SAM-Net [33] obtains better results by jointly learning semantic correspondence and attribute transfer. Huang et al [22] proposed DCCNet, which fuses correlation maps derived from local features and a context-aware semantic feature representation.…”

“…On the contrary, geometric and semantic matching architectures utilize global correlations [33,41,47,48,49], where similarities are evaluated between all pairs of locations in the dense feature maps. While capable of handling longrange matches, global correlation layers are computationally unfeasible at high resolutions.…”

Learning Accurate Dense Correspondences and When to Trust Them

“…Finding pixel-wise correspondences between pairs of images is a fundamental computer vision problem with numerous important applications, including dense 3D reconstruction [51], video analysis [44,57], image registration [55,62], image manipulation [15,37], and texture or style transfer [27,35]. Dense correspondence estimation has most commonly been addressed in the context of optical flow [2,17,22,60], where the image pairs represent consecutive frames in a video.…”