Single-Image Depth Prediction Makes Feature Matching Easier

Toft, Carl; Turmukhambetov, Daniyar; Sattler, Torsten; Kahl, Fredrik; Brostow, Gabriel J.

doi:10.1007/978-3-030-58517-4_28

Cited by 18 publications

(13 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Past research has shown that orthographic view generation improves performance in tasks such as image retrieval [5], and feature matching [6]. Monolayout [20] predicts an amodal scene layout using an encoder to extract features at multiple scales.…”

Section: B Orthographic View Generationmentioning

confidence: 99%

“…Monolayout [20] predicts an amodal scene layout using an encoder to extract features at multiple scales. Rectified features [6] performs perspective unwarping via the help of 3D information. It uses [21] to first predict a dense depth map and eventually clusters the surface normals for each point.…”

Section: B Orthographic View Generationmentioning

confidence: 99%

“…The use of local detectors and descriptors is widespread in computer and robotic vision problems that include descriptor matching between an image pair [1]- [4], relative pose estimation [2], [4], front-end pipelines for SLAM [5], and image retrievals [5], [6]. Traditional local descriptors [7], [8] typically employ a detector-descriptor pipeline; the detector comprises sparse or dense keypoints, while the descriptor characterizes a patch around the detector typically using a high dimensional vector.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching

Parihar

Gujarathi

Mehta

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

The use of local detectors and descriptors in typical computer vision pipelines work well until variations in viewpoint and appearance change become extreme. Past research in this area has typically focused on one of two approaches to this challenge: the use of projections into spaces more suitable for feature matching under extreme viewpoint changes, and attempting to learn features that are inherently more robust to viewpoint change. In this paper we present a novel framework that combines learning of invariant descriptors through data augmentation and orthographic viewpoint projection. We propose rotation-robust local descriptors, learnt through training data augmentation based on rotation homographies, and a correspondence ensemble technique that combines vanilla feature correspondences with those obtained through rotation-robust features. Using a range of benchmark datasets as well as contributing a new bespoke dataset for this research domain, we evaluate the effectiveness of the proposed approach on key tasks including pose estimation and visual place recognition. Our system outperforms a range of baseline and state-of-the-art techniques, including enabling higher levels of place recognition precision across opposing place viewpoints, and achieves practically-useful performance levels even under extreme viewpoint changes.

show abstract

Section: B Orthographic View Generationmentioning

confidence: 99%

Section: B Orthographic View Generationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching

Parihar

Gujarathi

Mehta

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…So far, we have either affine view synthesis (ASIFT [172], MODS [164]), or GAN-based stylizations [12] for the day-night matching. That is why I am glad to see papers like "Single-Image Depth Prediction Makes Feature Matching Easier" [260], which generate normalized views based on depth to help the matching. Why not go further?…”

Section: Matching With On-demand View Synthesis Revisitedmentioning

confidence: 99%

Learning and Crafting for the Wide Multiple Baseline Stereo

Mishkin¹

2021

Preprint

View full text Add to dashboard Cite

This thesis introduces the wide multiple baseline stereo (WxBS) problem. WxBS, a generalization of the standard wide baseline stereo problem, considers the matching of images that simultaneously differ in more than one image acquisition factor such as viewpoint, illumination, sensor type, or where object appearance changes significantly, e.g., over time. A new dataset with the ground truth, evaluation metric and baselines has been introduced.The thesis presents the following improvements of the WxBS pipeline. (i) A loss function, called HardNeg, for learning a local image descriptor that relies on hard negative mining within a mini-batch and on the maximization of the distance between the closest positive and the closest negative patches. (ii) The descriptor trained with the HardNeg loss, called HardNet, is compact and shows state-of-the-art performance in standard matching, patch verification and retrieval benchmarks. (iii) A method for learning the affine shape, orientation, and potentially other parameters related to geometric and appearance properties of local features. (iv) A tentative correspondences generation strategy which generalizes the standard first to second closest distance ratio is presented. The selection strategy, which shows performance superior to the standard method, is applicable to either hard-engineered descriptors like SIFT, LIOP, and MROGH or deeply learned like HardNet. (v) A feedback loop is introduced for the two-view matching problem, resulting in MODS -matching with on-demand view synthesis -algorithm. MODS is an algorithm that handles a viewing angle difference even larger than the previous state-of-the-art ASIFT algorithm, without a significant increase of computational cost over "standard" wide and narrow baseline approaches.Last, but not least, a comprehensive benchmark for local features and robust estimation algorithms is introduced. The modular structure of its pipeline allows easy integration, configuration, and combination of methods and heuristics. AbstraktTato práce představuje problém wide multiple baseline stereo (WxBS), který je zobecněním standardního problému wide baseline stereo. WxBS se zabývá párováním obrázků, které se současně liší více než v jednom faktoru získání obrázku, jako je stanoviště a zorný úhel, osvětlení, typ senzoru, nebo kde se vzhled objektu zásadně mění, např. v průběhu času. Byl představen nový dataset s anotacemi, evaluační metrikou a výchozími metodami.Disertační práce představuje následující vylepšení algoritmu WxBS: Ztrátovou funkci (i) pro učení lokálního deskriptoru obrázku založeného na výběru obtížných negativních příkladů v minibatchi a na maximalizaci rozdílu vzdálenosti od nejbližšího pozitivního a nejbližšího negativního příkladu. Výsledný deskriptor (ii), nazvaný HardNet, který je kompaktní a dosahuje vynikajících výsledků v standardním párování, ověřování malých částí obrázků a vyhledávání obrázků. Metodu pro učení afinního tvaru, orientace a potenciálně dalších parametrů souvisejících s geometrií a vzhledem lokálních příznaků (iii...

show abstract

“…In contrast to multi-scale pyramids, some works aim at being invariant to different scales through their learning process [9,33], however, as a side effect, they become progressively less discriminative [54]. Another possible direction, and popular strategy, is to estimate the local or global transformations and rectify the images prior to establishing the correspondences [29,35,49,57].…”

Section: Introductionmentioning

confidence: 99%

ScaleNet: A Shallow Architecture for Scale Estimation

Barroso-Laguna¹,

Tian²,

Mikolajczyk³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we address the problem of estimating scale factors between images. We formulate the scale estimation problem as a prediction of a probability distribution over scale factors. We design a new architecture, ScaleNet, that exploits dilated convolutions as well as selfand cross-correlation layers to predict the scale between images. We demonstrate that rectifying images with estimated scales leads to significant performance improvements for various tasks and methods. Specifically, we show how ScaleNet can be combined with sparse local features and dense correspondence networks to improve camera pose estimation, 3D reconstruction, or dense geometric matching in different benchmarks and datasets. We provide an extensive evaluation on several tasks, and analyze the computational overhead of ScaleNet. The code, evaluation protocols, and trained models are publicly available at https://github.com/axelBarroso/ScaleNet.

show abstract

Single-Image Depth Prediction Makes Feature Matching Easier

Cited by 18 publications

References 76 publications

RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching

RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching

Learning and Crafting for the Wide Multiple Baseline Stereo

ScaleNet: A Shallow Architecture for Scale Estimation

Contact Info

Product

Resources

About