The role of cross-modal associations in statistical learning

Glicksohn, Arit; Cohen, Asher

doi:10.3758/s13423-013-0458-4

Cited by 45 publications

(52 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a subsequent test stage, participants were more accurate in recognizing the complete audiovisual transition (1/A-2/B) than in recognizing transitions between auditory stimuli alone [1-2] or visual stimuli alone [A-B]. However, another study (Glicksohn & Cohen, 2013) conversely suggested that learning audiovisual associations is more demanding. Participants were presented with triplets of auditory [1-2-3], visual [A-B-C] or audiovisual [1/A-2/B-3/C] stimuli that formed fixed "words" presented sequentially amongst other such words.…”

Section: Limitations and Future Directionsmentioning

confidence: 98%

Visual cortex signals a mismatch between regularity of auditory and visual streams

2017

View full text Add to dashboard Cite

Understanding how humans code for and respond to environmental uncertainty/regularity is a question shared by current computational and neurobiological approaches to human cognition. To date, studies investigating neurobiological systems that track input uncertainty have examined responses to uni-sensory streams. It is not known, however, whether there exist brain systems that combine information about the regularity of input streams presented to different senses. We report an fMRI study that aimed to identify brain systems that relate statistical information across sensory modalities. We constructed temporally extended auditory and visual streams, each of which could be random or highly regular, and presented them concurrently. We found strong signatures of "regularity matching" in visual cortex bilaterally; responses were higher when the level of regularity in the auditory and visual streams mismatched than when it matched, [(AudHigh/VisLow and AudLow/VisHigh) > (AudLow/VisLow and AudHigh/VisHigh)]. In addition, several frontal and parietal regions tracked regularity of the auditory or visual stream independently of the other stream's regularity. An individual-differences analysis showed that signatures of single-modality-focused regularity tracking in these fronto-parietal regions are inversely related to signatures of regularity-matching in visual cortex. Our findings suggest that i) visual cortex is a junction for integration of temporally-extended auditory and visual inputs and that ii) multisensory regularity-matching depends on balanced processing of both input modalities. We discuss the implications of these findings for neurobiological models of uncertainty and for understanding computations that underlie multisensory interactions in occipital cortex.3

show abstract

Section: Limitations and Future Directionsmentioning

confidence: 98%

Visual cortex signals a mismatch between regularity of auditory and visual streams

2017

View full text Add to dashboard Cite

show abstract

“…The theoretical challenges that offline measures implicate are presented in figure 1, which outlines the components of individual performance in the classical visual SL (VSL) task (e.g. [3,17,36,75,76]; see [77] for a related approach).…”

Section: (B) Structural Confoundsmentioning

confidence: 99%

Towards a theory of individual differences in statistical learning

Siegelman

Bogaerts

Christiansen

et al. 2017

Phil. Trans. R. Soc. B

182

199

View full text Add to dashboard Cite

One contribution of 13 to a theme issue 'New frontiers for statistical learning in the cognitive sciences'. In recent years, statistical learning (SL) research has seen a growing interest in tracking individual performance in SL tasks, mainly as a predictor of linguistic abilities. We review studies from this line of research and outline three presuppositions underlying the experimental approach they employ: (i) that SL is a unified theoretical construct; (ii) that current SL tasks are interchangeable, and equally valid for assessing SL ability; and (iii) that performance in the standard forced-choice test in the task is a good proxy of SL ability. We argue that these three critical presuppositions are subject to a number of theoretical and empirical issues. First, SL shows patterns of modality-and informationalspecificity, suggesting that SL cannot be treated as a unified construct. Second, different SL tasks may tap into separate sub-components of SL that are not necessarily interchangeable. Third, the commonly used forced-choice tests in most SL tasks are subject to inherent limitations and confounds. As a first step, we offer a methodological approach that explicitly spells out a potential set of different SL dimensions, allowing for better transparency in choosing a specific SL task as a predictor of a given linguistic outcome. We then offer possible methodological solutions for better tracking and measuring SL ability. Taken together, these discussions provide a novel theoretical and methodological approach for assessing individual differences in SL, with clear testable predictions.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

show abstract

“…By analyzing recordings from wearable directional microphones attached to the heads of the participants who normally moved in various environments, Parise et al were able to conclude that the sources of the high-pitched sounds, in fact, tend to be elevated in the natural environment and, therefore, mapping between pitch and spatial height is likely to be grounded in this statistical regularity. The statistical regularities hypothesis and multisensory statistical learning, in particular, have received a lot of attention in the literature (e.g., Glicksohn & Cohen, 2013;Mitchel & Weiss, 2011); however, in many cases, the statistical origin of cross-modal associations is difficult to prove due to the need for substantial environment sampling. The hypothesis, at least theoretically, was extended to explain more complex second-order correspondences.…”

mentioning

confidence: 99%

Cross-Modal Associations between Color and Haptics

Slobodenyuk

Jraissati

Kanso

et al. 2015

Atten Percept Psychophys

View full text Add to dashboard Cite

The objective of the present study was to explore cross-modal associations between color and tactile sensation while using haptically rendered virtual stimuli with substance properties of roughness/smoothness, hardness/softness, heaviness/lightness, elasticity/inelasticity, and adhesiveness/ nonadhesiveness. The stimuli with the indicated properties were rendered with the aid of SensAble PHANTOM OMNI® haptic device. The experimental setup required the participants to use exploratory procedures typical to real object interaction, and select a color from the HSV color space that matched the experienced sensation. The findings of our investigation reveal systematic mapping between color characteristics and intensity of the haptic stimuli. Qualitatively different haptic sensations, however, produced relatively similar patterns of cross-modal associations.

show abstract

The role of cross-modal associations in statistical learning

Cited by 45 publications

References 39 publications

Visual cortex signals a mismatch between regularity of auditory and visual streams

Visual cortex signals a mismatch between regularity of auditory and visual streams

Towards a theory of individual differences in statistical learning

Cross-Modal Associations between Color and Haptics

Contact Info

Product

Resources

About