Professionally realistic multi-task environments, such as the NASA Multi-Attribute Task Battery (MATB-II), are employed for measuring vigilance and attention in professionals. However, it is unclear whether well-known intervention effects on performance during simple lab-based tasks, such as those of caffeine, also translate to these more realistic working situations. In a preregistered, double-blind, randomized, controlled repeated-measures experiment (https://osf.io/2zubx), we compared the performance of thirty-five civil pilots during vigilance- and attention-related tasks in simple (psychomotor vigilance task; auditory oddball detection) versus multitask environments (MATB-II system monitoring; MATB-II communications) after consuming regular vs. decaffeinated coffee. For vigilance tasks, no coffee intervention effects were found. Instead, a reversed task repetition effect was found, with participants being slower in session 2 in the simple task environment, but faster in session 2 in the complex environment. For attention-related tasks, regular coffee improved performance accuracy in the simple, but not the multitask environment. Coffee versus decaf effects in the simple task environment did not correlate with those in the complex task environment, neither for vigilance nor for selective attention. However, an experiment-wide increase in sleepiness was attenuated if participants drank regular coffee in the second session. This finding was supported by heart rate and eye blink measures. Results suggest that intervention-related findings do not easily translate to different vigilance- and attention-related tasks if task environments differ in complexity. The MATB-II multi-task environment, in its current form, is perhaps more suitable for assessing intervention effects on physiological measures of fatigue and vigilance than on cognitive performance.