Self-Localization of a Multi-Fisheye Camera Based Augmented Reality System in Textureless 3d Building Models

Urban, Steffen; Leitloff, Jens; Wursthorn, Sven; Hinz, Stefan

doi:10.5194/isprsannals-ii-3-w2-43-2013

Cited by 10 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatives to determine camera poses from imagery are model-based approaches [17,18]. Determining the camera pose in real time, in indoor environments, is practicable by CAD model matching [19][20][21][22]. Convolutional Neural Networks are used to determine matches between aerial images and UAV images [23] or terrestrial images and UAV images [24].…”

Section: Related Workmentioning

confidence: 99%

UAS Navigation with SqueezePoseNet—Accuracy Boosting for Pose Regression by Data Augmentation

Mueller

Jutzi

2018

Drones

View full text Add to dashboard Cite

Abstract:The navigation of Unmanned Aerial Vehicles (UAVs) nowadays is mostly based on Global Navigation Satellite Systems (GNSSs). Drawbacks of satellite-based navigation are failures caused by occlusions or multi-path interferences. Therefore, alternative methods have been developed in recent years. Visual navigation methods such as Visual Odometry (VO) or visual Simultaneous Localization and Mapping (SLAM) aid global navigation solutions by closing trajectory gaps or performing loop closures. However, if the trajectory estimation is interrupted or not available, a re-localization is mandatory. Furthermore, the latest research has shown promising results on pose regression in 6 Degrees of Freedom (DoF) based on Convolutional Neural Networks (CNNs). Additionally, existing navigation methods can benefit from these networks. In this article, a method for GNSS-free and fast image-based pose regression by utilizing a small Convolutional Neural Network is presented. Therefore, a small CNN (SqueezePoseNet) is utilized, transfer learning is applied and the network is tuned for pose regression. Furthermore, recent drawbacks are overcome by applying data augmentation on a training dataset utilizing simulated images. Experiments with small CNNs show promising results for GNSS-free and fast localization compared to larger networks. By training a CNN with an extended data set including simulated images, the accuracy on pose regression is improved up to 61.7% for position and up to 76.0% for rotation compared to training on a standard not-augmented data set.

show abstract

Section: Related Workmentioning

confidence: 99%

UAS Navigation with SqueezePoseNet—Accuracy Boosting for Pose Regression by Data Augmentation

Mueller

Jutzi

2018

Drones

View full text Add to dashboard Cite

show abstract

“…Step edges can be extracted with synthetic depth images and crease edges with normal images that can be both rendered from the model view (Urban et al, 2013). As it is shown in (Urban, 2016), the developed AR system outperforms all other systems in terms of reliability, robustness and flexibility/genericity.…”

Section: Augmented Reality Systemmentioning

confidence: 99%

“…The second is a self-localization method using the un-textured 3D building model. By comparing features from real and synthetic views of the environment, the system is able to estimate its pose w.r.t the building model either by using approximate nearest neighbour matching or particle filtering (Urban et al, 2013). These two requirements -on one hand recognizing re-visited scenes with the help of an improved online learning method for binary features (so-called "mdBrief" (Urban & Hinz, 2016)) and, on the other hand, finding the initial sensor position in a textureless model -led to the development of a model-supported multi-camera SLAM system called "MultiCol-SLAM" (Urban et al, 2016a).…”

Section: Augmented Reality Systemmentioning

confidence: 99%

Collaborative Multi-Scale 3d City and Infrastructure Modeling and Simulation

Breunig

Borrmann²,

Rank³

et al. 2017

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

Self Cite

View full text Add to dashboard Cite

Commission IV, WG IV/7KEY WORDS: Collaborative planning, integration of 3D modeling and simulation, multi-scale modelling, mobile computer vision. ABSTRACT:Computer-aided collaborative and multi-scale 3D planning are challenges for complex railway and subway track infrastructure projects in the built environment. Many legal, economic, environmental, and structural requirements have to be taken into account. The stringent use of 3D models in the different phases of the planning process facilitates communication and collaboration between the stake holders such as civil engineers, geological engineers, and decision makers. This paper presents concepts, developments, and experiences gained by an interdisciplinary research group coming from civil engineering informatics and geo-informatics banding together skills of both, the Building Information Modeling and the 3D GIS world. New approaches including the development of a collaborative platform and 3D multi-scale modelling are proposed for collaborative planning and simulation to improve the digital 3D planning of subway tracks and other infrastructures. Experiences during this research and lessons learned are presented as well as an outlook on future research focusing on Building Information Modeling and 3D GIS applications for cities of the future.

show abstract

“…Model-based approaches to determine camera poses from imagery are also available (Reitmayr and Drummond, 2006;Unger et al, 2016). Determining the camera pose in real time, in indoor environments, is practicable by CAD model matching (Ulrich et al, 2009;Zang and Hashimoto, 2011;Urban et al, 2013;Mueller and Voegtle, 2016). Convolutional neural networks are used to determine matches between aerial images and UAV images (Altwaijry et al, 2016) or terrestrial images and UAV images (Lin et al, 2015).…”

Section: Related Workmentioning

confidence: 99%

Squeezeposenet: Image Based Pose Regression With Small Convolutional Neural Networks for Real Time Uas Navigation

Müller

Urban

Jutzi

2017

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

Self Cite

View full text Add to dashboard Cite

ABSTRACT:The number of unmanned aerial vehicles (UAVs) is increasing since low-cost airborne systems are available for a wide range of users. The outdoor navigation of such vehicles is mostly based on global navigation satellite system (GNSS) methods to gain the vehicles trajectory. The drawback of satellite-based navigation are failures caused by occlusions and multi-path interferences. Beside this, local image-based solutions like Simultaneous Localization and Mapping (SLAM) and Visual Odometry (VO) can e.g. be used to support the GNSS solution by closing trajectory gaps but are computationally expensive. However, if the trajectory estimation is interrupted or not available a re-localization is mandatory. In this paper we will provide a novel method for a GNSS-free and fast image-based pose regression in a known area by utilizing a small convolutional neural network (CNN). With on-board processing in mind, we employ a lightweight CNN called SqueezeNet and use transfer learning to adapt the network to pose regression. Our experiments show promising results for GNSS-free and fast localization.

show abstract

Self-Localization of a Multi-Fisheye Camera Based Augmented Reality System in Textureless 3d Building Models

Cited by 10 publications

References 19 publications

UAS Navigation with SqueezePoseNet—Accuracy Boosting for Pose Regression by Data Augmentation

UAS Navigation with SqueezePoseNet—Accuracy Boosting for Pose Regression by Data Augmentation

Collaborative Multi-Scale 3d City and Infrastructure Modeling and Simulation

Squeezeposenet: Image Based Pose Regression With Small Convolutional Neural Networks for Real Time Uas Navigation

Contact Info

Product

Resources

About