Cross-modal conflicts in object recognition: determining the influence of object category

Abstract

Previous research examining cross-modal conflicts in object recognition has often made use of animal vocalizations and images, which may be considered natural and ecologically valid, thus strengthening the association in the congruent condition. The current research tested whether the same cross-modal conflict would exist for man-made object sounds as well as comparing the speed and accuracy of auditory processing across the two object categories. Participants were required to attend to a sound paired with a visual stimulus and then respond to a verification item (e.g., "Dog?"). Sounds were congruent (same object), neutral (unidentifiable image), or incongruent (different object) with the images presented. In the congruent and neutral condition, animals were recognized significantly faster and with greater accuracy than man-made objects. It was hypothesized that in the incongruent condition, no difference in reaction time or error rate would be found between animals and man-made objects. This prediction was not supported, indicating that the association between an object's sound and image may not be that disparate when comparing animals to man-made objects. The findings further support cross-modal conflict research for both the animal and man-made object category. The most important finding, however, was that auditory processing is enhanced for living compared to nonliving objects, a difference only previously found in visual processing. Implications relevant to both the neuropsychological literature and sound research are discussed.

Selective Enhancement of Object Representations through Multisensory Integration

“…Figure 2B shows that there was strong evidence (BF . 10) for faster RTs for animate objects compared with inanimate objects when presented in either the auditory or visual modalities, consistent with the results from previous studies (Murray et al, 2006;Yuval-Greenberg and Deouell, 2009;Vogler and Titchener, 2011;Carlson et al, 2014). However, there was inconclusive evidence (BF = 0.75) for the audiovisual condition.…”

“…In vision, animate objects offer substantial processing and perceptual advantages over inanimate objects, including being categorized faster, more consciously perceived, and found faster in search tasks (New et al, 2007;Jackson and Calvillo, 2013;Carlson et al, 2014;Ritchie et al, 2015;Lindh et al, 2019). Auditory studies have similarly found faster categorization times for animate objects (Yuval-Greenberg and Deouell, 2009;Vogler and Titchener, 2011). This difference may be a remnant of an evolutionary need to rapidly recognize and process living stimuli that could pose threats or be sources of sustenance (Laws, 2000).…”

Reward enhances cross‐modal conflict control in object categorization: Electrophysiological evidence

“…The congruency between stimuli from target and nontarget modalities can be defined according to the stimuli's temporal, spatial, and/or higher‐level (such as semantics) characteristics. Previous studies have demonstrated cross‐modal interference effects during object recognition (Diaconescu, Alain, & McIntosh, ; Molholm, Ritter, Javitt, & Foxe, ; Vogler & Titchener, ; Yuval‐Greenberg & Deouell, ). For example, in Yuval‐Greenberg and Deouell (), pairs of animal sounds and pictures were presented, and the sound and picture were associated with either the same animal (congruent) or different animal (incongruent).…”

Selective enhancement of object representations through multisensory integration

“…The animacy model was not significantly correlated to the auditory 7 RDM, implying that the animacy distinction is not as prominent in audition. In this study, we leveraged the visual and auditory encoding bias that has been observed for 5 animate objects over inanimate objects (Grootswagers, Ritchie, et al, 2017;Guerrero & Calvillo, 6 2016;Murray, 2006;Tzovara et al, 2012;Vogler & Titchener, 2011) to study how perceptual 7 biases across object categories influences multisensory enhancement in audiovisual objects. 8…”

“…In vision, animate objects 19 offer substantial processing and perceptual advantages over inanimate objects, including being: 20 categorized faster, more consciously perceived, and found faster in search tasks (Carlson et Ritchie et al, 2015). Auditory studies have similarly found faster 23 categorization times for animate objects (Vogler & Titchener, 2011;Yuval-Greenberg & 24 Deouell, 2009). This difference may be a remnant of an evolutionary need to rapidly recognize 25 and process living stimuli that could pose threats or be sources of sustenance (Laws, 2000).…”