Context dependency of time-based event-related expectations for different modalities

Abstract

Expectations about the temporal occurrence of events (when) are often tied with the expectations about certain event-related properties (what and where) happening at these time points. For instance, slowly waking up in the morning we expect our alarm clock to go off; however, the longer we do not hear it the more likely we already missed it. However, most current evidence for complex time-based event-related expectations (TBEEs) is based on the visual modality. Here we tested whether TBEEs can also act cross-modally. To this end, visual and auditory stimulus streams were presented which contained early and late targets embedded among distractors (to maximise temporal target uncertainty). Foreperiod-modality-contingencies were manipulated run-wise so that visual targets either occurred early in 80% of trials and auditory targets occurred late in 80 % of trials or vice versa. Participants showed increased sensitivity for expected auditory early/visual late targets which increased over time while the opposite pattern was observed for visual early/auditory late targets. A benefit in reaction times was only found for auditory early trials. Together, this pattern of results suggests that context-dependent TBEEs for auditory targets after short foreperiods (be they correct or not) dominated and determined which modality became more expected at the late position irrespective of the veridical statistical regularity. Hence, TBEEs in cross-modal and uncertain environments are context-dependent, shaped by the dominant modality in temporal tasks (i.e. auditory) and only boost performance cross-modally when expectations about the event after the short foreperiod match with the run-wise context (i.e. auditory early/visual late).

Minimal interplay between explicit knowledge, dynamics of learning and temporal expectations in different, complex uni- and multisensory contexts

“…As we argued in our recent paper, depending on whether stimuli compete for interest, the one better suited for the task (e.g., auditory stimuli in temporal tasks and visual stimuli in spatial tasks) potentially wins the race. Given the similarity of results across our studies (for further discussion, see Ball, Andreca, & Noesselt, 2021), TE and TBEE might be a single process (namely, always TBEE), in which the context simply determines for which target class temporal regularities are learned and how strong TEs are formed (e.g., unisensory visual paradigm: TE effect for visual; mixed paradigm with auditory and visual: TE(A) > TE(V)).…”

“…However, if the strength of expecting a certain modality at a certain point in time implies the presence of TBEEs, then unisensory TE paradigms are a special case of TBEEs (100% modality expectation). To investigate behavioural patterns of true multisensory TBEE (i.e., each foreperiod was equally likely), we recently conducted a study in which we only manipulated target-foreperiod contingencies (hence, TBEE) of auditory and visual targets (Ball, Andreca, & Noesselt, 2021). TBEEs were quantified as the performance difference between the primary and secondary targets (e.g., short foreperiod 80% auditory [primary] and 20% visual targets [secondary]) at each foreperiod.…”