Figure 1: Conceptual overview of our outdoor CMR system showing a detailed 3D model of a street that a local AR user can experience in collaboration with a remote VR user. The virtual coordinate space of the shared virtual environment is aligned to the real-world surroundings of the AR user (Space B) with QR code markers. Line illustrations by Suhyun Park (artist).

We are very interested in addressing the problem of building cityscale AR systems where users can travel anywhere at any time and see the correct graphics registered in the world around them. One crucial requirement for this is accurate tracking and localisation.In my work, I propose to tackle two themes. The first is to examine what good registration means in uncontrolled outdoor environments. The second is to explore how prior information can be used to support wide-area tracking efficiently and robustly.

In this paper, we propose a method to use semantic information to improve the use of map priors in a sparse, feature-based MonoSLAM system. To incorporate the priors, the features in the prior and SLAM maps must be associated with one another. Most existing systems build a map using SLAM and then align it with the prior map. However, this approach assumes that the local map is accurate, and the majority of the features within it can be constrained by the prior. We use the intuition that many prior maps are created to provide semantic information. Therefore, valid associations only exist if the features in the SLAM map arise from the same kind of semantic object as the prior map. Using this intuition, we extend ORB-SLAM2 using an open source pre-trained semantic segmentation network (DeepLabV3+) to incorporate prior information from Open Street Map building footprint data. We show that the amount of drift, before loop closing, is significantly smaller than that for original ORB-SLAM2. Furthermore, we show that when ORB-SLAM2 is used as a prior-aided visual odometry system, the tracking accuracy is equal to or better than the full ORB-SLAM2 system without the need for global mapping or loop closure.

Construction monitoring is vital for the timely delivery of projects. However, manual data collection and fusion methods are arduous. We propose a framework for autonomous multimodal data collection and VR visualisation. Based on "work-in-progress" results, we demonstrate its capabilities in-the-lab and validate its functionality on a real site. We explore how such a framework could complement construction-centric deep learning and 4D as-built datasets to aid human decision-making using VR.