The Effect of Sound on Visual Fidelity Perception in Stereoscopic 3-D

Abstract

Visual and auditory cues are important facilitators of user engagement in virtual environments and video games. Prior research supports the notion that our perception of visual fidelity (quality) is influenced by auditory stimuli. Understanding exactly how our perception of visual fidelity changes in the presence of multimodal stimuli can potentially impact the design of virtual environments, thus creating more engaging virtual worlds and scenarios. Stereoscopic 3-D display technology provides the users with additional visual information (depth into and out of the screen plane). There have been relatively few studies that have investigated the impact that auditory stimuli have on our perception of visual fidelity in the presence of stereoscopic 3-D. Building on previous work, we examine the effect of auditory stimuli on our perception of visual fidelity within a stereoscopic 3-D environment.

Effects of sound on visual realism perception and task performance

“…In contrast to the study that considered polygon count, sound consisting of music (classical or heavy metal) did not have any effect on the perception of visual realism when visual realism was defined with respect to texture resolution. These two experiments were repeated in a later study but the visuals were presented in stereoscopic 3D [19]. Visual realism perception with stereoscopic 3D viewing increased in the presence of music (classical and heavy metal), but decreased in the presence of white noise.…”

“…Our previous studies focused on examining the perception of visual realism of static 3D rendered (virtual) scenes under various auditory conditions [17][18][19][20]. The purpose of those studies was to determine whether sound can lead to an increase in visual realism perception and allow us to employ lower realism visuals and therefore reduce computational load.…”

“…Our own work examined visual realism perception in the presence of various auditory conditions within a static environment [17][18][19][20]. In our first two studies, the visuals consisted of a single 2D image of a surgeon's head (a rendered 3D model).…”

“…For each of the visuals, their task was to rate the realism of the presented visual (model) on a scale from 1 to 7. With respect to polygon count, visual realism perception increased in the presence of classical music, particularly when considering images corresponding to higher polygon count [17]. When considering visual realism defined with respect to texture resolution, sound consisting of white noise had very specific and detrimental effects.…”

“…Our more recent work considered the effect of sound cues that were related to the visual scene on visual realism perception, defined with respect to texture resolution [20]. The visual stimuli consisted of static images of a surgeon (upper torso and head) holding a surgical drill, where, in each image the texture resolution of the model representing the surgeon varied.…”