Minimal interplay between explicit knowledge, dynamics of learning and temporal expectations in different, complex uni- and multisensory contexts

Abstract

While temporal expectations (TE) generally improve reactions to temporally predictable events, it remains unknown how temporal rule learning and explicit knowledge about temporal rules contribute to performance improvements and whether any contributions generalise across modalities. Here, participants discriminated the frequency of diverging auditory, visual or audiovisual targets embedded in auditory, visual or audiovisual distractor sequences. Temporal regularities were manipulated run-wise (early vs. late target within sequence). Behavioural performance (accuracy, RT) plus measures from a computational learning model all suggest that temporal rule learning occurred but did not generalise across modalities, that dynamics of learning (size of TE effect across runs) and explicit knowledge have little to no effect on the strength of TE, and that explicit knowledge affects performance - if at all - in a context dependent manner: only under complex task regimes (unknown target modality) might it partially help to resolve response conflict while it is lowering performance in less complex environments.

Cross-modal perceptual enhancement of unisensory targets is uni-directional and does not affect temporal expectations

“…The results are mainly in line with our previous research showing that the learning of temporal regularities can be context-dependent and occurs with a higher likelihood for multisensory than unisensory targets (Ball et al, 2020;Ball, Andreca, et al, 2021;Ball, Fuehrmann, et al, 2018;Ball, Michels, et al, 2018;Ball, Spuerck, et al, 2021). Given the absence of TE effects in the unisensory conditions, we also replicated our findings that there seems to be an interactive effect of TEs and multisensory interactions, at least when induced by redundant targets.…”

“…However, this cross-modal enhancement of unisensory target perception in fast stimulation sequences appears to depend on the asynchronous presentation of irrelevant stimuli. Note that our previous studies showed that synchronously presented irrelevant stimuli decreased unisensory target perception (Ball, Fuehrmann, et al, 2018;Ball, Michels, et al, 2018;Ball, Spuerck, et al, 2021). Together these studies suggest that while synchronously presented irrelevant stimuli likely distracted participants, asynchronous presentation resultedat least partially (visual only)in crossmodal enhancement.…”

“…Here we additionally show that the multisensory TE effects (enhancement of response times and perceptual sensitivity) can occur for stimulation sequences with frequencies of up to 15 Hz. Note that in our previous studies (Ball, Fuehrmann, et al, 2018;Ball, Michels, et al, 2018;Ball, Spuerck, et al, 2021), we used a 5 Hz stimulus train leaving the possibility (although unlikely) that participants did not direct attention to a certain point in time but rather to a certain position in the stimulus sequence (i.e. they counted).…”

“…were enlarged in multisensory compared to unisensory conditions (Ball, Michels, et al, 2018;Ball, Fuehrmann, et al, 2018;Ball, Spuerck, et al, 2021). However, it is currently unknown whether only redundant target presentations (synchronously presented audio-visual targets) or also other forms of multisensory interplay (here the perceptual enhancement of targets by asynchronously flanked irrelevant stimuli) can cause such synergistic effects.…”

“…While spatial proximity might be an important factor governing multisensory interaction under some circumstances (see e.g. Frassinetti et al, 2002), it may not be a necessary requirement for all types of multisensory interplay (for review and discussion see Spence, 2013), especially in temporal and non-spatial tasks (Ball, Michels, et al, 2018;Ball, Fuehrmann, et al, 2018;Ball, Spuerck, et al, 2021;Noesselt et al, 2005).…”