This manuscript identifies and documents unsolved problems and research challenges in the extended reality (XR) domain (i.e., virtual (VR), augmented (AR), and mixed reality (MR)). The manuscript is structured to include technology, design, and human factor perspectives. The text is visualization/display-focused, that is, other modalities such as audio, haptic, smell, and touch, while important for XR, are beyond the scope of this paper. We further narrow our focus to mainly geospatial research, with necessary deviations to other domains where these technologies are widely researched. The main objective of the study is to provide an overview of broader research challenges and directions in XR, especially in spatial sciences. Aside from the research challenges identified based on a comprehensive literature review, we provide case studies with original results from our own studies in each section as examples to demonstrate the relevance of the challenges in the current research. We believe that this paper will be of relevance to anyone who has scientific interest in extended reality, and/or uses these systems in their research.

No abstract

Structure from motion (SfM) has emerged as a popular method for characterising marine benthos (seabed organisms), particularly in clear, tropical waters. However, there are many environmentally sensitive benthic organisms inhabiting temperate waters, including the reef‐forming glass sponges of the north‐east Pacific Ocean. Broader questions are raised, not just about whether SfM is a capable spatial data acquisition and ecological characterisation method in temperate waters; but whether a systematic assessment of capture methods in dry and wet laboratory conditions reveals critical relationships between SfM parameters, data products and their implications for underwater surveys. This paper, the first of two empirical assessments, reports on a series of dry‐lab tests quantifying the impact that lighting, camera type, camera settings and capture strategy have on data accuracy. These tests provide a crucial accuracy baseline for subsequent wet‐lab and field‐based surveys, revealing that photographs captured from a controlled and stable platform produce superior data products. While the measurable differences were small, they may be critical for accurate change detection in temperate environments.

Advancements in extended reality (XR) have inspired new uses and users of advanced visualization interfaces, transforming geospatial data visualization and consumption by enabling interactive 3D geospatial data experiences in 3D. Conventional metrics (e.g., mental rotations test (MRT)) are often used to assess and predict the appropriateness of these visualizations without accounting for the effect the interface has on those metrics. We developed the Immersive MRT (IMRT) to evaluate the impact that virtual reality (VR) based visualizations and 3D virtual environments have on mental rotation performance. Consistent with previous work, the results of our pilot study suggest that mental rotation tasks are performed more accurately and rapidly with stereo 3D stimuli than with 2D images of those stimuli.

Abstract. Structure-from-motion (SfM) has emerged as a popular method of characterizing marine benthos in tropical marine environments and could be of tremendous value to glass sponge monitoring and management efforts in the Northeast Pacific Ocean. However, temperate marine environments present a unique set of challenges to SfM workflows, and the combined impact that cold, dark, and turbid waters have on the veracity of SfM derived data must be critically evaluated in order for SfM to become a meaningful tool for ongoing glass sponge research. This paper discusses the design, development, testing, and deployment of an innovative underwater SfM workflow for generating high-resolution 3D models in temperate marine environments. This multi-phase research project (dry-lab, wet-lab, and field), while possibly seen as unconventional, was designed to innovate in two ways. First to build an operational data capture platform to support low-cost SfM-based seafloor surveys. And second, to enable systematic isolation and evaluation of SfM data capture parameters and their implications for representational veracity and data quality. This paper reports the challenges and outcomes from a series of field surveys conducted in Howe Sound, BC, one of which serves as the first of two data sets in a temporal analysis of 3D morphometric change. This research demonstrates that accurate, high-resolution morphometric characterization, of all benthic species and habitats, is dependent on a range of equipment, procedural, and environmental variables. It is also intended to share our applied problem-solving path to successful 3D capture, backed up by robust data science.