The Hologram in My Hand: How Effective is Interactive Exploration of 3D Visualizations in Immersive Tangible Augmented Reality?

Abstract

We report on a controlled user study comparing three visualization environments for common 3D exploration. Our environments differ in how they exploit natural human perception and interaction capabilities. We compare an augmented-reality head-mounted display (Microsoft HoloLens), a handheld tablet, and a desktop setup. The novel head-mounted HoloLens display projects stereoscopic images of virtual content into a user's real world and allows for interaction in-situ at the spatial position of the 3D hologram. The tablet is able to interact with 3D content through touch, spatial positioning, and tangible markers, however, 3D content is still presented on a 2D surface. Our hypothesis is that visualization environments that match human perceptual and interaction capabilities better to the task at hand improve understanding of 3D visualizations. To better understand the space of display and interaction modalities in visualization environments, we first propose a classification based on three dimensions: perception, interaction, and the spatial and cognitive proximity of the two. Each technique in our study is located at a different position along these three dimensions. We asked 15 participants to perform four tasks, each task having different levels of difficulty for both spatial perception and degrees of freedom for interaction. Our results show that each of the tested environments is more effective for certain tasks, but that generally the desktop environment is still fastest and most precise in almost all cases.

The Impact of Immersion on Cluster Identification Tasks

“…As shown in previous research, the degree of immersion can have an effect on spatial cognition and memorability in various contexts [11,33]. Some studies even indicate correlations between the degree of immersion and efficiency in cluster identification, distance estimation, and outlier detection tasks in scatterplot visualizations [4,41]. However, many existing studies use a variety of different interaction techniques individually for each design space, disguising possible effects caused solely by characteristics of the different design spaces.…”

“…Over the last few years, augmented-, virtual-, and mixed-reality (AR, VR, MR) hardware and software have been on the rise, opening up new design spaces for visual analytics (VA) applications. Various examples of visualizations exist in VR, AR, and MR, either restricting the visualization's space to a small area [4] or allowing it to occupy the entire space around the observer [14,30]. As the level of immersion with regard to the visualization differs largely between the two kinds, their visualization design spaces can be seen as two individual ones.…”

“…It is often not a trivial decision which design space is best suited for a specific task. There are several studies comparing visualizations in VREs to those in conventional design spaces, but they often focus on differences resulting from different visualization and interaction techniques [4,58]. These studies do not capture how much of the differences in performance can be ascribed to those two factors and how much to the different levels of immersion.…”

Dataspace: A Reconfigurable Hybrid Reality Environment for Collaborative Information Analysis

“…Whereas previous environments have mostly dealt with visual exploration of scientific data, the unique characteristics of Dataspace make it a good candidate for flexible focus-and-context analysis of multiple types of information at the same time -as demonstrated by the applications presented in the previous section. In particular, Dataspace bridges complementary visualization environments and allows users to seamlessly switch between them, providing a wider support for the different perceptual and interaction tasks that characterize visual information analysis [7].…”

Multiple Views: different meanings and collocated words

“…The user can raise their hand to view the information, hence the phrase view on a bat [Rob07]. Augmented reality and visualizations is another growth area [BSB∗18], and in fact many researchers are working towards moving visualisation away from the desktop environment [RRB∗14]. These concepts are infrequently discussed in visualisation texts (see Table 7).…”