Implications of Audio and Narration in the User Experience Design of Virtual Reality

The Role of Audio in Immersive Storytelling: a Systematic Review in Cultural Heritage

“…An asymmetric co-occurrence matrix is created by calculating the concepts' relative co-occurrence frequencies. This matrix produces a two-dimensional concept map using a proprietary clustering algorithm based on Audio First [152][153][154][155][156] Audio and Multimodal Integration [157][158][159][160][161][162][163][164][165][166] Audio in Multimodal Comparison [167] Multimodal Experience [168][169][170][171][172][173][174][175] the spring-force model for the many-body problem [83]. The correctness of each concept in this semantic network generates a third hierarchical dimension, which displays the more general parent concepts at the higher levels.…”

“…Two of them use different eye-tracking techniques to detect the user's gaze and present information about the artwork or artistic building the user is looking at in a virtual environment. In particular, Kelling et al [167] uses eye position estimation based on an IMU sensor of an HMD, Sanchez et al [130] uses gaze to control the playback of audio elements, such as effect or noises, while reading an children's tales book, and Kwok et al [113] use eye-tracking glasses. In a fourth study [158], one of the author's interactive installations uses voice and pitch tracking to induce vibration in objects inside a virtual environment based on voice intensity.…”

“…To foster a more natural interaction, several works [112,134,153,160,169] have adopted custom-built tangible devices that are usually used as the main element during the interactive experience and are usually rated as engaging by users. Despite the few works reviewed, we would like to mention eye tracking as another promising opportunity to (i) provide information in an ecological manner during interaction with the virtual environment (e.g., [113,167]), (ii) make the experience accessible in the presence of users with physical impairments( [135]), (iii) augment the experience with multisensory elements such as the environmental sound of tactile feedback (e.g., [130]).…”

The hierarchy of needs for user experiences in virtual reality

“…In a previous evaluation of the initial prototype, several issues with the experience of the application were identified, including issues with navigation, poor image quality, and lack of engagement (Kauhanen et al 2017). In further iterations of the prototype, these issues were addressed and improved, especially so with the addition of audio and narration (Kelling et al 2018). The current study further examines the effectiveness of the improvements and also dives deeper into the complexities of the immersive experience in an attempt to provide insight for researchers and content creators to utilize and build upon.…”