Optimal Multi-View Surface Normal Estimation Using Affine Correspondences

Baráth, Dániel; Eichhardt, Ivan; Hajder, Levente

doi:10.1109/tip.2019.2895542

Cited by 10 publications

(11 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Application: surface normal estimation using affine correspondences. We applied the multi-view least-squares optimal method from [8] to estimate surface normals from the extracted and corrected LAFs. The used sequences from the Strecha dataset are fountain -p11, herzjesus-p8 and herzjesus-p25 since those are the only ones with publicly available ground truth 3D point cloud.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Optimal Multi-view Correction of Local Affine Frames

Eichhardt,

Barath

2019

Preprint

Self Cite

View full text Add to dashboard Cite

The technique requires the epipolar geometry to be pre-estimated between each image pair. It exploits the constraints which the camera movement implies, in order to apply a closed-form correction to the parameters of the input affinities. Also, it is shown that the rotations and scales obtained by partially affine-covariant detectors, e.g. AKAZE or SIFT, can be completed to be full affine frames by the proposed algorithm. It is validated both in synthetic experiments and on publicly available real-world datasets that the method always improves the output of the evaluated affine-covariant feature detectors. As a by-product, these detectors are compared and the ones obtaining the most accurate affine frames are reported. For demonstrating the applicability, we show that the proposed technique as a pre-processing step improves the accuracy of pose estimation for a camera rig, surface normal and homography estimation.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Therefore, if the cameras are calibrated, the normal can be unambiguously estimated from a single correspondence [15]. Multiple-view normal estimation [8,12] is also possible. Pritts et al [26,27] showed that the radial distortion parameters can be retrieved, as well.…”

Section: Introductionmentioning

confidence: 99%

Optimal Multi-view Correction of Local Affine Frames

Eichhardt,

Barath

2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Ivan Eichhardt and Levente Hajder have a series of works exploiting the local affine correspondences for surface normals estimation [24]…”

Section: Surface Normals Estimationmentioning

confidence: 99%

Learning and Crafting for the Wide Multiple Baseline Stereo

Mishkin¹

2021

Preprint

View full text Add to dashboard Cite

This thesis introduces the wide multiple baseline stereo (WxBS) problem. WxBS, a generalization of the standard wide baseline stereo problem, considers the matching of images that simultaneously differ in more than one image acquisition factor such as viewpoint, illumination, sensor type, or where object appearance changes significantly, e.g., over time. A new dataset with the ground truth, evaluation metric and baselines has been introduced.The thesis presents the following improvements of the WxBS pipeline. (i) A loss function, called HardNeg, for learning a local image descriptor that relies on hard negative mining within a mini-batch and on the maximization of the distance between the closest positive and the closest negative patches. (ii) The descriptor trained with the HardNeg loss, called HardNet, is compact and shows state-of-the-art performance in standard matching, patch verification and retrieval benchmarks. (iii) A method for learning the affine shape, orientation, and potentially other parameters related to geometric and appearance properties of local features. (iv) A tentative correspondences generation strategy which generalizes the standard first to second closest distance ratio is presented. The selection strategy, which shows performance superior to the standard method, is applicable to either hard-engineered descriptors like SIFT, LIOP, and MROGH or deeply learned like HardNet. (v) A feedback loop is introduced for the two-view matching problem, resulting in MODS -matching with on-demand view synthesis -algorithm. MODS is an algorithm that handles a viewing angle difference even larger than the previous state-of-the-art ASIFT algorithm, without a significant increase of computational cost over "standard" wide and narrow baseline approaches.Last, but not least, a comprehensive benchmark for local features and robust estimation algorithms is introduced. The modular structure of its pipeline allows easy integration, configuration, and combination of methods and heuristics. AbstraktTato práce představuje problém wide multiple baseline stereo (WxBS), který je zobecněním standardního problému wide baseline stereo. WxBS se zabývá párováním obrázků, které se současně liší více než v jednom faktoru získání obrázku, jako je stanoviště a zorný úhel, osvětlení, typ senzoru, nebo kde se vzhled objektu zásadně mění, např. v průběhu času. Byl představen nový dataset s anotacemi, evaluační metrikou a výchozími metodami.Disertační práce představuje následující vylepšení algoritmu WxBS: Ztrátovou funkci (i) pro učení lokálního deskriptoru obrázku založeného na výběru obtížných negativních příkladů v minibatchi a na maximalizaci rozdílu vzdálenosti od nejbližšího pozitivního a nejbližšího negativního příkladu. Výsledný deskriptor (ii), nazvaný HardNet, který je kompaktní a dosahuje vynikajících výsledků v standardním párování, ověřování malých částí obrázků a vyhledávání obrázků. Metodu pro učení afinního tvaru, orientace a potenciálně dalších parametrů souvisejících s geometrií a vzhledem lokálních příznaků (iii...

show abstract

“…A feature-based SfMapproach with a novel approach that refines motion (camera parameters and poses) and structure (both 3D points and surface normals) demonstrates the use of ACs for 3D reconstruction. Related publications are [6,1,12,13,14,16,17,18,19,4].…”

Section: Main Contributionsmentioning

confidence: 99%

Affine Correspondences and Challenges in Multi-Sensor Systems

Eichhardt¹

2020

View full text Add to dashboard Cite

Multi-device systems of cameras and various depth sensors are widely used these days in the industry. Some can also operate well in conditions where others cannot (e.g. active laser sensors compared to cameras). Various views and sensors of different modalities reveal valuable information about the environment, which is crucial for the robust operation of, e.g., detection or decision-making algorithms. The fusion of sensor data, calibration and multiple-view geometry are critical topics discussed in this dissertation.Sensors are usually placed in a common frame of reference determined by intrinsic and other parameters describing sensor alignment. During the process of calibration, all parameters can be estimated and tuned based on correspondences between sensor views. The mainstream computer vision methods solving geometric tasks use corresponding points across views as an input. Based on the correspondences, it is possible to estimate the underlying geometry of the views (i.e., epipolar geometry). As a next step, the 3D structure of the scene can also be determined. Corresponding image points are established as the extracted centers of dominant image regions. However, the shape and orientation of the regions also contain useful information as they are related to the underlying surface. Only a minority of the computer vision community tries to utilize this inter-region relation, mostly constraining their approaches to the basic pinhole camera model. The first-order approximation of a region correspondence is called an Affine Correspondence (AC). Sensor fusion is also an important part of modern systems that observe and analyze the environment. Sensors of distinct views and different modalities (e.g., depth and color images) complement each other. A low-resolution Time-of-Flight (ToF) depth camera image can be supersampled using a high-resolution color image of the same view as guidance, while depth may provide e.g. a posterior option for refocusing the color view.This thesis provides a thorough investigation of ACs in the theoretical and algorithmic sense to obtain rapid, more robust and high-quality geometric model estimation in twoor multiple-view cases. It is demonstrated that ACs are usable in general scenarios where real-world cameras and diverse geometry complicate the task. Next, the data-level sensor fusion of high-resolution color-and lower-resolution ToF depth-cameras is investigated for single pairs of frames and also for video sequences. Finally, the calibration of multi-sensor systems is discussed, that include LiDARs and cameras (wide-angle, fisheye optics, etc.). i I would like to express my gratitude to my colleagues at Institute for Computer Science and Control (SZTAKI) and at Eötvös Loránd University (ELTE) whom I had the pleasure to work with, especially, to my advisor Dmitry Chetverikov for introducing me to image processing and computer vision, guiding me throughout my Ph.D. study and research, and all his support, knowledge and critical view on research he shared with me. I would like to tha...

show abstract

Optimal Multi-View Surface Normal Estimation Using Affine Correspondences

Cited by 10 publications

References 26 publications

Optimal Multi-view Correction of Local Affine Frames

Optimal Multi-view Correction of Local Affine Frames

Learning and Crafting for the Wide Multiple Baseline Stereo

Affine Correspondences and Challenges in Multi-Sensor Systems

Contact Info

Product

Resources

About