Recent dual-task studies observed worse performance in task-pair switches than in task-pair repetitions and interpreted these task-pair switch costs as evidence that the identity of the two individual tasks performed within a dual task is jointly represented in a single mental representation, termed “task-pair set.” In the present study, we conducted two experiments to examine (a) whether task-pair switch costs are due to switching cues or/and task pairs and (b) at which time task-pair sets are activated during dual-task processing. In Experiment 1, we used two cues per task-pair and found typical dual-task interference, indicating that performance in the individual tasks performed within the dual task deteriorates as a function of increased temporal task overlap. Moreover, we observed cue switch costs, possibly reflecting perceptual cue priming. Importantly, there were also task-pair switch costs that occur even when controlling for cue switching. This suggests that task-pair switching per se produces a performance cost that cannot be reduced to costs of cue switching. In Experiment 2, we employed a go/no-go-like manipulation and observed task-pair switch costs after no-go trials where subjects prepared for a task-pair, but did not perform it. This indicates that task-pair sets are activated before performing a dual task. Together, the findings of the present study provide further evidence for a multicomponent hierarchical representation consisting of a task-pair set organized at a hierarchically higher level than the task sets of the individual tasks performed within a dual task.