Effects of the Simultaneous Presentation of Corresponding Auditory and Visual Stimuli on Size Variance Perception

Abstract

To overcome limitations in perceptual bandwidth, humans condense various features of the environment into summary statistics. Variance constitutes indices that represent diversity within categories and also the reliability of the information regarding that diversity. Studies have shown that humans can efficiently perceive variance for visual stimuli; however, to enhance perception of environments, information about the external world can be obtained from multisensory modalities and integrated. Consequently, this study investigates, through two experiments, whether the precision of variance perception improves when visual information (size) and corresponding auditory information (pitch) are integrated. In Experiment 1, we measured the correspondence between visual size and auditory pitch for each participant by using adjustment measurements. The results showed a linear relationship between size and pitch—that is, the higher the pitch, the smaller the corresponding circle. In Experiment 2, sequences of visual stimuli were presented both with and without linked auditory tones, and the precision of perceived variance in size was measured. We consequently found that synchronized presentation of audio and visual stimuli that have the same variance improves the precision of perceived variance in size when compared with visual-only presentation. This suggests that audiovisual information may be automatically integrated in variance perception.

Variance aftereffect within and between sensory modalities for visual and auditory domains

“…In contrast, variance is an index representing the degree of homogeneity of the stimuli. Although less attention has been paid to variance perception compared to average perception, in recent years increasing evidence has been reported on people also being able to perceive the variance in various visual elements-from lower-level stimuli such as orientation (Morgan et al, 2008;Solomon, 2010), luminance (Tong et al, 2015), motion (Suárez-Pinilla et al, 2018), size (Solomon et al, 2011;Ueda et al, 2018), and color (Bronfman et al, 2014;Maule & Franklin, 2020;Michael et al, 2014;Ward et al, 2016) to higher-level stimuli such as facial expressions (Haberman et al, 2015) and race (Phillips et al, 2018). The duration time of stimulus presentation in the previous studies was approximately 150-1,000 ms, indicating that observers can perceive variance in a short time.…”

“…Moreover, we also tested whether the adaptation aftereffect of variance perception occurred across different modalities. Variance extraction has unique characteristics that differ from the mean, which is a highly common statistic across different stimuli attributes and sensory modalities (Maule & Franklin, 2020;Ueda et al, 2014Ueda et al, , 2018. For example, when there are different-sized oranges and matchsticks lined up in different orientations, it is nonsensical to compare the average size with that of the orientation.…”

“…Average perception is possible in a temporal context (Albrecht et al, 2012 ; Albrecht & Scholl, 2010 ; Haberman et al, 2009 ; Hubert-Wallander & Boyton, 2015 ; Yamanashi Leib et al, 2016 ), and observers can perceive averages in temporal or sequential presentations as accurately as in spatial or simultaneous presentation (Chong & Treisman, 2005 ). Regarding variance perception, few studies have focused on the variance of stimuli presented in a sequence (Payzan-LeNestour et al, 2016 ; Ueda et al, 2015 , 2018 ). In particular, Payzan-LeNestour et al ( 2016 ) asked participants to rate the magnitude of perceived volatility of movement trajectories or numerosity that changed over time on a 5-point scale and demonstrated the variance adaptation effect.…”

“…Given the characteristics of variance as a statistical indicator that is common to different sensory modalities, it is possible to extend the domain-general variance perception mechanism to cross-modality. Ueda et al ( 2018 ) examined the effect of audition on visual variance perception using auditory stimuli with a pitch corresponding to the visual size. It was shown that synchronized presentation of auditory and visual stimuli that have the same variance improves the precision of perceived variance in size, when compared with the condition of only visual stimuli presentation.…”