HDD-Net: Hybrid Detector Descriptor with Mutual Interactive Learning

Barroso-Laguna, Axel; Verdie, Yannick; Busam, Benjamin; Mikolajczyk, Krystian

doi:10.1007/978-3-030-69525-5_30

Cited by 15 publications

(9 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…R2D2 [12] identifies keypoints as reliable and repeatable positions in image and trains reliability through AP loss [31]. HDD-Net [32] weights the features with softargmax scores in grids to train the score map and feature map simultaneously. Furthermore, DISK [33] and reinforced SP [34] relax the keypoint detection and descriptor matching as probabilistic processes and train the network with reinforcement learning.…”

Section: B Score Map Based Methodsmentioning

confidence: 99%

ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction

Zhao¹,

Wu²,

Miao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Existing methods detect the keypoints in a nondifferentiable way, therefore they can not directly optimize the position of keypoints through back-propagation. To address this issue, we present a differentiable keypoint detection module, which outputs accurate sub-pixel keypoints. The reprojection loss is then proposed to directly optimize these sub-pixel keypoints, and the dispersity peak loss is presented for accurate keypoints regularization. We also extract the descriptors in a sub-pixel way, and they are trained with the stable neural reprojection error loss. Moreover, a lightweight network is designed for keypoint detection and descriptor extraction, which can run at 95 frames per second for 640×480 images on a commercial GPU. On homography estimation, camera pose estimation, and visual (re-)localization tasks, the proposed method achieves equivalent performance with the state-of-the-art approaches, while greatly reduces the inference time.

show abstract

Section: B Score Map Based Methodsmentioning

confidence: 99%

ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction

Zhao¹,

Wu²,

Miao³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

“…Visual robustness has been the focus of numerous works in the field of correspondence search [4,12,23]. The rotation has been addressed by correcting input patches [12,22,29] before extracting local descriptors [27,47,48], or by designing robust architectures [4,21,23]. However, in the context of scale changes, the standard strategy is the well-known multi-scale (M-S) pyramid approach, which applies methods at different re-scaled versions of the image [10,36,46].…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, the scale change is arguably the most challenging, and the most important parameter to estimate compared to rotation, translation, or even local affine deformations [53]. There are several strategies to deal with scale changes, with the multi-scale pyramid being one of the most popular solutions [3,4,10,22,36,51,53]. Although the multi-scale pyramid mitigates the problem of different scales, it increases complexity and ambiguity as the matcher needs to establish correspondences among multiple scale levels.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ScaleNet: A Shallow Architecture for Scale Estimation

Barroso-Laguna¹,

Tian²,

Mikolajczyk³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we address the problem of estimating scale factors between images. We formulate the scale estimation problem as a prediction of a probability distribution over scale factors. We design a new architecture, ScaleNet, that exploits dilated convolutions as well as selfand cross-correlation layers to predict the scale between images. We demonstrate that rectifying images with estimated scales leads to significant performance improvements for various tasks and methods. Specifically, we show how ScaleNet can be combined with sparse local features and dense correspondence networks to improve camera pose estimation, 3D reconstruction, or dense geometric matching in different benchmarks and datasets. We provide an extensive evaluation on several tasks, and analyze the computational overhead of ScaleNet. The code, evaluation protocols, and trained models are publicly available at https://github.com/axelBarroso/ScaleNet.

show abstract

“…To tackle geometry deformation induced by scale and viewpoint variation across images, tremendous efforts have been made within local feature matching pipelines. Methods directly performing convolution upon multi-scale images such as KeyNet [3] and HDDNet [5], or implicitly applying multi-scale detection such as ASLfeat [23] and DenseNet [21] are intended to mimic conventional scale space theory. Other approaches tend to mitigate scale and viewpoint change by predicting a co-visible area [35], estimating the scale distribution [4], or simply regressing warping deformation [6,24].…”

Section: Introductionmentioning

confidence: 99%

Adaptive Assignment for Geometry Aware Local Feature Matching

Huang¹,

Chen²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Local image feature matching, aiming to identify and correspond similar regions from image pairs, is an essential concept in computer vision. Most existing image matching approaches follow a one-toone assignment principle and employ mutual nearest neighbor to guarantee unique correspondence between local features across images. However, images from different conditions may hold large-scale variations or viewpoint diversification so that one-to-one assignment may cause ambiguous or missing representations in dense matching. In this paper, we introduce AdaMatcher, a novel detector-free local feature matching method, which first correlates dense features by a lightweight feature interaction module and estimates co-visible area of the paired images, then performs a patch-level many-to-one assignment to predict match proposals, and finally refines them based on a one-to-one refinement module. Extensive experiments show that AdaMatcher outperforms solid baselines and achieves state-of-the-art results on many downstream tasks. Additionally, the many-to-one assignment and one-to-one refinement module can be used as a refinement network for other matching methods, such as SuperGlue, to boost their performance further. Code will be available upon publication.

show abstract

HDD-Net: Hybrid Detector Descriptor with Mutual Interactive Learning

Cited by 15 publications

References 39 publications

ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction

ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction

ScaleNet: A Shallow Architecture for Scale Estimation

Adaptive Assignment for Geometry Aware Local Feature Matching

Contact Info

Product

Resources

About