Verbal and novel multisensory associative learning in adults

Abstract

To date, few studies have focused on the behavioural differences between the learning of multisensory auditory-visual and intra-modal associations. More specifically, the relative benefits of novel auditory-visual and verbal-visual associations for learning have not been directly compared. In Experiment 1, 20 adult volunteers completed three paired associate learning tasks: non-verbal novel auditory-visual (novel-AV), verbal-visual (verbal-AV; using pseudowords), and visual-visual (shape-VV). Participants were directed to make a motor response to matching novel and arbitrarily related stimulus pairs. Feedback was provided to facilitate trial and error learning. The results of Signal Detection Theory analyses suggested a multisensory enhancement of learning, with significantly higher discriminability measures (d-prime) in both the novel-AV and verbal-AV tasks than the shape-VV task. Motor reaction times were also significantly faster during the verbal-AV task than during the non-verbal learning tasks. Experiment 2 (n = 12) used a forced-choice discrimination paradigm to assess whether a difference in unisensory stimulus discriminability could account for the learning trends in Experiment 1. Participants were significantly slower at discriminating unisensory pseudowords than the novel sounds and visual shapes, which was notable given that these stimuli produced superior learning. Together the findings suggest that verbal information has an added enhancing effect on multisensory associative learning in adults

The Interplay Between Multisensory Associative Learning and IQ in Children

“…Consistent with the findings of Fifer et al. (), children and adults performed best on the verbal‐AV task, but unlike adults, children did not show a generalized multisensory advantage for novel multisensory stimuli. Although, all three associative learning tasks significantly correlated with IQ, only the d‐prime measure on the verbal‐visual associative task was identified as a unique predictor of FSIQ scores, suggesting that general intellectual abilities in children may be related to multisensory verbal associative learning abilities.…”

“…The generalized novel multisensory learning advantage (relative to unisensory incidental learning) found in adults by Fifer et al. () was not evident in children. This finding is consistent with other studies that have shown immaturities in different types of multisensory learning, such as visual‐haptic learning and incidental learning of auditory‐visual object categories, throughout childhood (Broadbent et al., ; Pirogovsky, Murphy, & Gilbert, ).…”

“…However, there are limitations inherent in a protocol of this type when the aim is to assess arbitrary associations using novel stimuli that are not readily verbalized. We have extended on these findings in adults and demonstrated a phonetic verbal‐visual associative learning advantage over the learning of visual stimuli that were paired with sounds that could not be pronounced (Fifer et al., ). Indeed visual‐verbal learning (even without semantics) represents an important variant of multisensory learning; verbal labeling of objects and events is evolutionarily an important function, frequently required, particularly in children, as new concepts and words are learnt (Gogate, Walker‐Andrews, & Bahrick, ; Sloutsky & Napolitano, ).…”

“…See Figure B and Fifer et al. () to view all visual and auditory stimuli used, and the experimental design.…”

“…Within a typical school environment there is an abundance of facilitatory multisensory information (e.g., auditory and visual speech information, and written and spoken instruction), in addition to distracting stimuli (e.g., background noise and movement). Indeed multisensory processes have been shown to enhance the speed and accuracy of responses (Barutchu, Crewther, & Crewther, ; Barutchu et al., ; Jordan & Baker, ; Mahoney, Verghese, Dumas, Wang, & Holtzer, ; Miller, ), as well as improve learning and memory throughout the life span (Botta et al., ; Broadbent, White, Mareschal, & Kirkham, ; Brunetti, Indraccolo, Mastroberardino, Spence, & Santangelo, ; Fifer, Barutchu, Shivdasani, & Crewther, ; Heikkila & Tiippana, ; Lehmann & Murray, ; Matusz, Wallace, & Murray, ; Shams & Seitz, ; Seitz, Kim, & Shams, ). Multisensory training has also been shown to enhance unisensory learning (e.g., Alais & Cass, ; Seitz et al., ).…”