Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer.

Fig. 1. We present ScanGAN360, a generative adversarial approach to scanpath generation for 360 • images. For a given 360 • scene, ScanGAN360 generates realistic scanpaths (center ), outperforming state-of-the-art methods (left) and mimicking the human baseline (right).

We measure detection thresholds for lateral translation gains of virtual camera motion in response to the corresponding head motion under natural viewing. We propose three applications for our method, addressing three key problems in virtual reality, and validate their improvement when our manipulation technique is applied.

a reduced physical space, and (iii) simulator sickness may be alleviated in simple scenarios when our compression method is applied.CCS Concepts: • Human-centered computing → Virtual reality.

No abstract