Hybrid Touch/Tangible Spatial Selection in Augmented Reality

Abstract

We study tangible touch tablets combined with Augmented Reality Head‐Mounted Displays (AR‐HMDs) to perform spatial 3D selections. We are primarily interested in the exploration of 3D unstructured datasets such as cloud points or volumetric datasets. AR‐HMDs immerse users by showing datasets stereoscopically, and tablets provide a set of 2D exploration tools. Because AR‐HMDs merge the visualization, interaction, and the users' physical spaces, users can also use the tablets as tangible objects in their 3D space. Nonetheless, the tablets' touch displays provide their own visualization and interaction spaces, separated from those of the AR‐HMD. This raises several research questions compared to traditional setups. In this paper, we theorize, discuss, and study different available mappings for manual spatial selections using a tangible tablet within an AR‐HMD space. We then study the use of this tablet within a 3D AR environment, compared to its use with a 2D external screen.

Investigating the Effect of Operation Mode and Manifestation on Physicalizations of Dynamic Processes

“…The co-located study also indicates positive social engagement when groups interact with a physical display together. Sereno et al [SGBI22] compare different tangible interaction methods for the selection of data points in 3D space. Their findings indicate that AR devices have a lesser impact on subjective workload compared to screen-based methods.…”

Working with Forensic Practitioners to Understand the Opportunities and Challenges for Mixed-Reality Digital Autopsy

“…Immersive analytics scenarios can be implemented in a variety of contexts, ranging from handheld mobile devices providing e.g., see-through augmented reality, to head-worn devices or caves [7,10]. Many research projects have sought to utilise and apply immersive analytics techniques in a variety of other research disciplines ranging from fluid dynamics and flows [8,17,40], surgery planning and medical data [13,57], molecular and microscopic data visualisation [71], or astrophysics and particle physics [64,74]. In our work, we propose to implement a prototype of immersive analytics for forensic autopsies which relies heavily on 3D data analysis and would therefore likely benefit from immersive environments.…”

Challenges of intracellular visualization using virtual and augmented reality

“…In particular, while the stereoscopic view of the data is highly valued, the HoloLens' default input mode of manual gestures in 3D space is not ideal for the intended application. Therefore, other forms of input should be explored, such as tablets (Sereno et al, 2022a) (Figure 2C, right image and Supplementary Video S2) or VR controllers in gameoriented headsets. Why should cell biologists now be interested in these techniques?…”

“…For example, this replication was applied to the fine-tuning of filter modules ( Figure 2A , part 2), which requires not only precise input but also observation of the module and of changes in the 3D view as well as quantitative histogram representation ( Figure 2B ; right image). AR can also exploit tablet technology ( Sereno et al, 2022a ) as illustrated here ( Figure 2C ; left image input data set as viewed in the napari viewer; right image, selected structures as a function of volume indicated by the color map, viewed on tablet using MorphoNetAR; Supplementary Video S2 ) using data launched in the web browser MorphoNet ( Leggio et al, 2019 ).…”