Marker-Less AR in the Hybrid Room Using Equipment Detection for Camera Relocalization

Rodas, Nicolas Loy; Barrera, Fernando; Padoy, Nicolas

doi:10.1007/978-3-319-24553-9_57

Cited by 7 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Camera pose estimation is a key computer vision problem, with applications in simultaneous localisation and mapping (SLAM) [1], [2], [3], [4], virtual and augmented reality [5], [6], [7], [8], [9], [10] and navigation [11]. In SLAM, the camera pose is commonly initialised upon starting reconstruction and then tracked from frame to frame, but tracking can easily be lost due to e.g.…”

Section: Introductionmentioning

confidence: 99%

Real-Time RGB-D Camera Pose Estimation in Novel Scenes Using a Relocalisation Cascade

Cavallari¹,

Golodetz²,

Lord³

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Camera pose estimation is an important problem in computer vision, with applications as diverse as simultaneous localisation and mapping, virtual/augmented reality and navigation. Common techniques match the current image against keyframes with known poses coming from a tracker, directly regress the pose, or establish correspondences between keypoints in the current image and points in the scene in order to estimate the pose. In recent years, regression forests have become a popular alternative to establish such correspondences. They achieve accurate results, but have traditionally needed to be trained offline on the target scene, preventing relocalisation in new environments. Recently, we showed how to circumvent this limitation by adapting a pre-trained forest to a new scene on the fly. The adapted forests achieved relocalisation performance that was on par with that of offline forests, and our approach was able to estimate the camera pose in close to real time, which made it desirable for systems that require online relocalisation. In this paper, we present an extension of this work that achieves significantly better relocalisation performance whilst running fully in real time. To achieve this, we make several changes to the original approach: (i) instead of simply accepting the camera pose hypothesis produced by RANSAC without question, we make it possible to score the final few hypotheses it considers using a geometric approach and select the most promising one; (ii) we chain several instantiations of our relocaliser (with different parameter settings) together in a cascade, allowing us to try faster but less accurate relocalisation first, only falling back to slower, more accurate relocalisation as necessary; and (iii) we tune the parameters of our cascade, and the individual relocalisers it contains, to achieve effective overall performance. Taken together, these changes allow us to significantly improve upon the performance our original state-of-the-art method was able to achieve on the well-known 7-Scenes and Stanford 4 Scenes benchmarks. As additional contributions, we present a novel way of visualising the internal behaviour of our forests, and use the insights gleaned from this to show how to entirely circumvent the need to pre-train a forest on a generic scene.

show abstract

Section: Introductionmentioning

confidence: 99%

Real-Time RGB-D Camera Pose Estimation in Novel Scenes Using a Relocalisation Cascade

Cavallari¹,

Golodetz²,

Lord³

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…A preliminary version has been presented in [13], which is hereby extended in the following ways. First, a complete description of the system and of the tracking method is provided.…”

Section: B Systems For Improving Radiation Safety In the Ormentioning

confidence: 99%

“…These, however, often require precise manual initializations and are limited to static scenes and to a set of viewpoints. Recently, AR has been introduced as a potential solution to help increase the awareness of clinicians to the potentially harmful ionizing radiation which is generated in the hybrid operating room during minimally invasive X-ray guided procedures [11]- [13]. Following this line, we hereby propose a mobile AR system (figure 1), where information related to radiation safety is displayed directly in the user's view through a hand-held screen or a head-mounted display (HMD).…”

Section: Introductionmentioning

confidence: 99%

See It With Your Own Eyes: Markerless Mobile Augmented Reality for Radiation Awareness in the Hybrid Room

Rodas

Barrera

Padoy

2017

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

show abstract

“…Visual re-localization is the problem of estimating the precise position and orientation from which a given image was taken with respect to a known scene. It is a key component of advanced computer vision applications such as AR/VR [8,22,42,65,73,77,101], and robotics systems such as self-driving cars [19] and drones [63]. Real-world scenes are highly dynamic, exhibiting changes in illumination, appearance and/or geometry.…”

Section: Introductionmentioning

confidence: 99%

Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes

Wald¹,

Sattler²,

Golodetz³

et al. 2020

Preprint

View full text Add to dashboard Cite

Long-term camera re-localization is an important task with numerous computer vision and robotics applications. Whilst various outdoor benchmarks exist that target lighting, weather and seasonal changes, far less attention has been paid to appearance changes that occur indoors. This has led to a mismatch between popular indoor benchmarks, which focus on static scenes, and indoor environments that are of interest for many real-world applications. In this paper, we adapt 3RScan -a recently introduced indoor RGB-D dataset designed for object instance re-localization -to create RIO10, a new long-term camera re-localization benchmark focused on indoor scenes. We propose new metrics for evaluating camera re-localization and explore how state-of-the-art camera re-localizers perform according to these metrics. We also examine in detail how different types of scene change affect the performance of different methods, based on novel ways of detecting such changes in a given RGB-D frame. Our results clearly show that long-term indoor re-localization is an unsolved problem. Our benchmark and tools are publicly available at waldjohannau.github.io/RIO10.

show abstract

Marker-Less AR in the Hybrid Room Using Equipment Detection for Camera Relocalization

Cited by 7 publications

References 11 publications

Real-Time RGB-D Camera Pose Estimation in Novel Scenes Using a Relocalisation Cascade

Real-Time RGB-D Camera Pose Estimation in Novel Scenes Using a Relocalisation Cascade

See It With Your Own Eyes: Markerless Mobile Augmented Reality for Radiation Awareness in the Hybrid Room

Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes

Contact Info

Product

Resources

About