International workshop on multimodal virtual and augmented reality (workshop summary)

Abstract

Virtual reality (VR) and augmented reality (AR) are expected by many to become the next wave of computing with significant impacts on our daily lives. Motivated by this, we organized a workshop on "Multimodal Virtual and Augmented Reality (MVAR)" at the 18 th ACM International Conference on Multimodal Interaction (ICMI 2016). While current VR and AR installations mostly focus on the visual domain, we expect multimodality to play a crucial role in future, next generation VR/AR systems. The submissions for this workshop reflect the potential of multimodality for VR and AR, illustrate interesting new directions, and pinpoint important issues. This paper gives a short motivation for the workshop and its aim, and summarizes the aforementioned trends and challenges identified from the submissions. CCS Concepts• Human-centered computing~Mixed / augmented reality KeywordsMultimodal virtual reality; multimodal mixed reality; multimodal augmented reality. MOVTIVATIONVirtual reality (VR) and augmented reality (AR) are currently two of the "hottest" topics in the IT industries. Many consider them to be the next wave in computing with a similar impact as the shift from desktop systems to mobiles and wearables. The related hype is rooted in technological improvements (such as mass-produced high-resolution mobile displays), which resulted in cheaper, highperformant devices ready for the common consumer market. Examples include Facebook's Oculus Rift and Microsoft's HoloLens for VR and AR scenarios, respectively.Despite this progress, we are still far from the ultimate goal of creating new virtual environments or augmentations of existing ones that feel and react similarly as their real counterparts. Many challenges and open research questions remain -mostly in the areas of multimodality and interaction. For example, current setups predominantly focus on visual and auditory senses, neglecting other modalities such as touch and smell that are an integral part of how we experience the real world around us. Likewise, it is still an open question how to best interact and communicate with a virtual world or virtual objects in AR. Multimodal interaction offers great potential to not only make this experience more realistic, but also to provide more powerful and efficient means of interacting with virtual and augmented worlds.In order to address related questions and to identify current trends and future challenges, we organized a workshop on "Multimodal Virtual and Augmented Reality (MVAR)" at the 18 th ACM International Conference on Multimodal Interaction (ICMI 2016). The aim of this workshop was to investigate any aspects about multimodality and multimodal interaction in relation to VR and AR. What are the most pressing research questions? What are the challenges? What opportunities do other modalities than vision offer for VR and AR? What are new and better ways for interaction with virtual objects and for an improved experience of VR and AR worlds? We invited researchers, practitioners, and visionaries to contribute thei...

Influence of Passive Haptic and Auditory Feedback on Presence and Mindfulness in Virtual Reality Environments

“…Though participants assumed a positive relation between such modalities and mindfulness, adding them could also contradict the derived theme of simplicity. As a starting point, we refer to the overview by Hürst et al [27], as well as work by Maggioni et al [41] for olfactory, Sheehan et al [69] for thermoceptory, and Obrist [50] for gustatory implementation possibilities for VR. We hope that this paper will inspire further inquiry into how the interplay of different modalities in VR can support mindfulness practice and eventually increases the well-being of users.…”

Digital Smell: Toward Electrically Reproducing Artificial Smell Sensations

“…In HCI, VR, and AR the sense of smell is used to deliver multisensory experiences and sometimes to alter other sensory inputs [5], [9], [26], [27], [28]. Multisensory technologies and their applications in terms of interaction, design, and challenges have been discussed recently in many places [29], [30], [31], [32], [33]. To avoid the limitations of the chemical-based smell delivery systems, finding alternative methods that can effectively reproduce olfactory sensations without chemicals is becoming a necessity.…”

Electric Smell: Towards Electrically Reproducing Artificial Smell Sensations

“…In HCI, VR, and AR the sense of smell is used to deliver multisensory experiences and sometimes to alter other sensory inputs [26], [27], [9], [5], [28]. Multisensory technologies and their applications interms of interaction, design, and challenges have been discussed recently in many places [29], [30], [31], [32], [33]. To avoid the pitfalls of the chemical based smell delivery systems, finding alternative methods that can effectively reproduce olfactory sensations without chemicals is becoming a necessity.…”