There is an unresolved discrepancy between popular hierarchical and multiple-demand perspectives on the functional organisation of the human frontal lobes. Here, we tested alternative predictions of these perspectives with a novel fMRI switching paradigm. Each trial involved switching attention between stimuli, but at different levels of difficulty and abstraction. As expected, increasing response times were evident when comparing low-level perceptual switching to more abstract dimension, rule and task-switching. However, there was no evidence of an abstraction hierarchy within the prefrontal cortex (PFC). Nor was there recruitment of additional anterior PFC regions under increased switching demand. Instead, switching activated a widespread network of frontoparietal and cerebellar regions. Critically, the activity within PFC sub-regions uniformly increased with behavioural switch costs. We propose that both perspectives have some validity, but neither is complete. Too many studies have reported dissociations within MD for this volume to be functionally uniform, and the recruitment of more anterior regions with increased general difficulty cannot explain those results. Conversely, whilst reproducible evidence for a hierarchical functional organisation has been reported, this cannot be explained in terms of abstraction of representation or reconfiguration per se, because those interpretations generalise poorly to other task contexts.