Interactions in Perceived Quality of Auditory-Visual Displays

Abstract

The quality of realism in virtual environments (VEs) is typically considered to be a function of visual and audio fidelity mutually exclusive of each other. However, the VE participant, being human, is multimodal by nature. Therefore, in order to validate more accurately the levels of auditory and visual fidelity that are required in a virtual environment, a better understanding is needed of the intersensory or crossmodal effects between the auditory and visual sense modalities. To identify whether any pertinent auditory-visual cross-modal perception phenomena exist, 108 subjects participated in three experiments which were completely automated using HTML, Java, and JavaScript programming languages. Visual and auditory display quality perceptions were measured intraand intermodally by manipulating the pixel resolution of the visual display and Gaussian white noise level, and by manipulating the sampling frequency of the auditory display and Gaussian white noise level. Statistically significant results indicate that high-quality auditory displays coupled with highquality visual displays increase the quality perception of the visual displays relative to the evaluation of the visual display alone, and that low-quality auditory displays coupled with high-quality visual displays decrease the quality perception of the auditory displays relative to the evaluation of the auditory display alone. These findings strongly suggest that the quality of realism in VEs must be a function of both auditory and visual display fidelities inclusive of each other.

Sound design and perception in walking interactions

“…Most prior work in this area has focused on sound delivery methods (Storms and Zyda, 2000;Sanders and Scorgie, 2002), sound quantity and quality of auditory versus visual information (Chueng and Marsden, 2002) and 3D sound (Freeman and Lessiter, 2001;Vastfjall, 2003). Recent studies have investigated the role of auditory cues in enhancing selfmotion and presence in VEs (Larsson et al, 2004;Kapralos et al, 2004;Väljamäe et al, 2005).…”

Sound synthesis and evaluation of interactive footsteps for virtual reality applications

“…Most prior work in this area has focused on sound delivery methods [25,24], sound quantity and quality of auditory versus visual information [4] and 3D sound [11,28]. Recent studies have investigated the role of auditory cues in enhancing self-motion and presence in virtual environments [17,15,26].…”

Sensory effect dataset and test setups

“…The setup of the test environment (i.e., location) is mainly based on standardized procedures such as the ITU-T Recommendation P.910 [16] or ITU-R Recommendation BT.500-11 [17]. Furthermore, we also based the setup on the paper from Storms et al [20] which describes in detail a test setup for a subjective quality assessment. Note that most of the video sequences from the presented dataset (cf.…”