Objectively measuring drivers’ emotions in real-world conditions is a challenging endeavour. This study investigated whether drivers’ emotional responses were captured more meaningfully by unimodal measurements or by a multimodal machine-learning approach. Ten participants drove a 23-mile route around Sunnyvale, California, while their heart rate, breathing rate and facial expressions were recorded. At regular intervals, participants indicated how they were feeling. After the study, independent observers reviewed a sample of the videotaped sessions, classifying the participants as experiencing high or low levels of stress according to their behaviour. The degree to which drivers’ self-report scores, single physiological data streams and facial behaviour - as judged by a facial recognition machine classifier - reflected their actual stress levels was compared to the multimodal algorithmic estimates which combined physiological and facial data outputs. The results showed that, compared to the other single data modes, the multimodal approach captured how the participants were feeling in a way which most meaningfully corresponded with their observed behaviour. The findings re-affirm the need for multimodal emotion-recognition systems for capturing driver stress. The limitations of the study are also discussed.