Crossmodal Effects in Task Switching: Modality Compatibility with Vocal and Pedal Responses

Abstract

Modality compatibility refers to the similarity between the stimulus modality and the modality of response-related sensory consequences (e.g., vocal output produces audible effects). While previous studies found higher costs of task switching with stimulus-response modality-incompatible tasks (auditory-manual and visual-vocal), the present study was aimed to explore the generality of modality compatibility by examining a new response modality (pedal responses). Experiment 1 showed that the effect of modality compatibility generalizes to pedal responses when these replaced manual responses used in previous studies (i.e., higher switch costs when switching between auditory-pedal and visual-vocal tasks compared to switching between auditory-vocal and visual-pedal tasks). However, in single-task conditions there was no influence of modality compatibility. Experiment 2 was designed to examine whether modality compatibility depends on the frequency of task switches. To this end, one task occurred very frequently, overall decreasing the task switching frequency. Importantly, the results showed a robust task-switching benefit of modality-compatible mappings even for a highly frequent task, suggesting that the sustained representation of potentially competing response modalities affects task-switching performance independent from the actual frequency of the tasks. Together, the data suggest that modality compatibility is an emergent phenomenon arising in task-switching situations based on the necessity to maintain but at the same time separate competing modality mappings, which are characterized by ideomotor ‘‘backward’’ linkages between anticipated response effects and the stimuli that called for this response in the first place.

Dissociating stimulus-response compatibility and modality compatibility in task switching

“…All analyses were calculated at α = .05. Data were collapsed across the two modality-incompatible tasks versus the two modality-compatible tasks, respectively, as in previous studies (e.g., Stephan et al, 2013 , 2021 ; Stephan & Koch, 2010 , 2011 , 2015 , 2016 ). Since MC is defined as an interaction of stimulus modality and response modality, collapsing the data is necessary to turn MC into a factor in the first place, rather than analysing trivial influences of different stimulus modalities and response modalities separately.…”

“…Thus, studies that only varied stimulus modality or response modality, or did so across blocks rather than within blocks, cannot answer the question of whether S-R compatibility and MC interact. In other words, most studies to date on S-R compatibility did not manipulate MC, and in turn, preceding studies on MC in task switching (Fintor et al, 2018a , b , 2020 ; Stephan et al, 2013 , 2021 ; Stephan & Koch, 2010 , 2011 , 2015 , 2016 ) did not manipulate spatial S-R compatibility in a systematic way. Note though that for dual-tasking, Stelzel and Schubert ( 2011 ) varied S-R compatibility, finding no interaction between MC and what they called “categorical crosstalk” (p. 481), that is, congruency effects between the modality mappings, since visual stimuli required a left/right and auditory stimuli a low/high judgment.…”

“…In several studies, Stephan and Koch ( 2010 , 2011 ) demonstrated that MC affects switch costs, showing larger switch costs in blocks with two modality-incompatible mappings than in blocks with two modality compatible mappings (Fintor et al, 2020 , 2018a , b ; Stephan & Koch, 2010 , 2011 , 2015 , 2016 ; Stephan et al, 2013 , 2021 ; see also Göthe et al, 2016 ; Hazeltine et al, 2006 , Schacherer & Hazeltine, 2020 ; Wirth et al, 2020 , for related findings in dual-task research). This MC effect on switch costs has been attributed to larger between-task crosstalk with incompatible modality mappings (Stephan & Koch, 2011 ).…”

The State-of-the-art of Research into Human Multitasking: An Editorial

“…Therefore, it is the "cream of the crop" discipline that requires effective cognitive control to minimize the costs of task interference. Stephan et al (2021) examined the costs of task switching with stimulus-response modality-incompatible tasks (auditory-manual and visual-vocal) depending on the generality of modality compatibility and the frequency of task switches. The frequency of switches did not influence the switching costs compared to modality-compatible tasks but modality-compatibility effects generalized from manual to pedal responses.…”