Efficient Linear Attention for Fast and Accurate Keypoint Matching

Suwanwimolkul, Suwichaya; Komorita, Satoshi

doi:10.1145/3512527.3531369

Cited by 7 publications

(2 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the advent of the deep learning era, the robustness of local features under illumination variation and viewpoint change can be significantly improved by learning-based methods, such as SuperPoint [27], D2-Net [28], R2D2 [29], and AWDesc [30]. In addition to learning-based detector-descriptors, some works recently focus on learning local feature matching [6], [20], [21], [23], [24], [31]. As a pioneering work, SuperGlue [6] requires two sets of keypoints and their visual descriptors as input and learns their correspondences using an attentional GNN with complete graphs over keypoints within and across images.…”

Section: Previous Arts a Local Feature Matchingmentioning

confidence: 99%

“…Despite the inspiring progress, these methods are inherently limited by the following two aspects: 1) Their inputs are putative correspondences provided by an NN search, so it is impossible to retrieve correct matches beyond these ones, which means that the upper bound on the performance is limited by the quality of initial matches; 2) Local visual descriptors and correspondence coordinates are separately used for match generation and mismatch rejection, respectively, neglecting the interaction between visual information from high-level image representation and geometric information from 2D keypoint distribution, calling for the research towards the design of effective feature matching methods to break the limitation of vanilla nearest neighbor correspondences. To this end, a line of research directly learns to find the partial assignment between two sets of local features with an attention-based Graph Neural Network (GNN) [20], [21]. Representatively, SuperGlue [6] constructs densely-connected graphs over intra-and inter-image keypoints and performs message passing by means of self-and cross-attention.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

HetGNN-SF: Self-supervised learning on heterogeneous graph neural network via semantic strength and feature similarity

Liu

Yan

et al. 2023

Appl Intell

View full text Add to dashboard Cite

Accurately matching local features between a pair of images corresponding to the same 3D scene is a challenging computer vision task. Previous studies typically utilize attentionbased graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images for visual and geometric information reasoning. However, in the context of feature matching, a significant number of keypoints are non-repeatable due to occlusion and failure of the detector, and thus irrelevant for message passing. The connectivity with non-repeatable keypoints not only introduces redundancy, resulting in limited efficiency (quadratic computational complexity w.r.t. the keypoint number), but also interferes with the representation aggregation process, leading to limited accuracy. Targeting towards high accuracy and efficiency, we propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide compact and meaningful message passing. More specifically, our Bilateral Context-Aware Sampling (BCAS) Module first dynamically samples two small sets of well-distributed keypoints with high matchability scores from the image pair. Then, our Matchable Keypoint-Assisted Context Aggregation (MKACA) Module regards sampled informative keypoints as message bottlenecks and thus constrains each keypoint only to retrieve favorable contextual information from intra-and inter-matchable keypoints, evading the interference of irrelevant and redundant connectivity with non-repeatable ones. Furthermore, considering the potential noise in initial keypoints and sampled matchable ones, the MKACA module adopts a matchability-guided attentional aggregation operation for purer data-dependent context propagation. By these means, we achieve the state-of-the-art performance on relative camera estimation, fundamental matrix estimation, and visual localization, while significantly reducing computational and memory complexity compared to typical attentional GNNs.

show abstract

Section: Previous Arts a Local Feature Matchingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

HetGNN-SF: Self-supervised learning on heterogeneous graph neural network via semantic strength and feature similarity

Liu

Yan

et al. 2023

Appl Intell

View full text Add to dashboard Cite

show abstract

Learning accurate template matching with differentiable coarse-to-fine correspondence refinement

Gao¹,

Yi²,

Zheng³

et al. 2024

Comp. Visual Media

View full text Add to dashboard Cite

Template matching is a fundamental task in computer vision and has been studied for decades. It plays an essential role in manufacturing industry for estimating the poses of different parts, facilitating downstream tasks such as robotic grasping. Existing methods fail when the template and source images have different modalities, cluttered backgrounds, or weak textures. They also rarely consider geometric transformations via homographies, which commonly exist even for planar industrial parts. To tackle the challenges, we propose an accurate template matching method based on differentiable coarse-to-fine correspondence refinement. We use an edge-aware module to overcome the domain gap between the mask template and the grayscale image, allowing robust matching. An initial warp is estimated using coarse correspondences based on novel structure-aware information provided by transformers. This initial alignment is passed to a refinement network using references and aligned images to obtain sub-pixel level correspondences which are used to give the final geometric transformation. Extensive evaluation shows that our method to be significantly better than state-of-the-art methods and baselines, providing good generalization ability and visually plausible results even on unseen real data.

show abstract

FMAP: Learning robust and accurate local feature matching with anchor points

Dai,

Xie,

Wang

et al. 2024

Expert Systems with Applications

View full text Add to dashboard Cite

Efficient Linear Attention for Fast and Accurate Keypoint Matching

Abstract: Figure 1: Our method versus the bigger SOTAs-SuperGlue and SGMNet-on speed (left), image matching * (top), and 3D reconstruction (bottom).

Cited by 7 publications

References 32 publications

HetGNN-SF: Self-supervised learning on heterogeneous graph neural network via semantic strength and feature similarity

HetGNN-SF: Self-supervised learning on heterogeneous graph neural network via semantic strength and feature similarity

Learning accurate template matching with differentiable coarse-to-fine correspondence refinement

FMAP: Learning robust and accurate local feature matching with anchor points

Contact Info

Product

Resources

About