Assessing Driving Simulator Validity: A Comparison of Multi-Modal Smartphone Interactions across Simulated and Field Environments

Abstract

The use of a driving simulator as a tool to evaluate secondary task performance elicits the question of simulator validity. After upgrading an existing driving simulator from a medium-fidelity to a high-fidelity configuration with a new software environment, a study was run to benchmark this simulator against previously published highway-driving data. A primary goal was to assess relative and absolute validity in a simulated highway environment. Data from 72 participants who performed manual and voice-based contact dialing tasks with a center-stack-mounted smartphone in either the driving simulator or driving on the highway in one of two vehicles is considered. This analysis compared secondary task demand between the simulator and on-road vehicles by primarily considering driver off-road glance behavior. Mean total eyes-off-road time, mean single-glance duration, and the number of long off-road glances showed similar patterns relative to the manual versus voice-based tasks in the simulator and the two on-road vehicles. A driving performance metric, percentage change of standard deviation of velocity, showed differing results between the simulator and on-road vehicles. It is concluded that these data make a strong argument for relative validity, and in some cases absolute validity, for this simulator for studying glance behavior associated with in-vehicle devices under a highway-driving scenario.

Underload on the Road: Measuring Vigilance Decrements During Partially Automated Driving

“…On the other hand, simulations may also be less arousing for the driver than real-world driving, so researchers must be aware of this when conducting future simulation research. Furthermore, simulations cannot create the same risks as real-world driving, so drivers may respond differently to safety critical situations than they would on the road (Caird and Horrey, 2011), though see McWilliams et al (2018) for examples when performance in simulated driving corresponds to real-world driving.…”

“…Table of papers that investigate driver vigilance during PAD.Parentheses after critical events indicates the number of critical events used per drive. On-Road = real-world highway driving, SimL = low fidelity driving simulator, SimM = medium fidelity driving simulator, SimH = high fidelity driving simulator (seeMcWilliams et al, 2018 for details). ∼ estimate of drive length when not directly reported.…”

Driver performance under simulated and actual driving conditions: Validity and orthogonality

“…Mullen, Charlton, Devlin & Bédard (2011), attempt to answer precisely this question by reviewing thirty-three studies, some threequarters of which were published in peer-reviewed journals, in which some form of validation was attempted. Many of these studies, together with more recent examples (McWilliams, Ward, Mehler & Reimer, 2018), seek to establish whether specific changes that occur during simulated driving also occur during actual driving (e.g. effect of a secondary task), on a single aspect of driving (e.g.…”

Driving Simulator Validation for In-Vehicle Human Machine Interface Assessment

“…The data used to make these comparisons can come from driver performance in terms of vehicle control, task performance, driver physiology, workload ratings, or glance behavior (Blaauw, 1982;McWilliams et al, 2018;Reimer et al, 2006;Reimer et al, 2011;Wang et al, 2010;Yan, Adbel-Aty, Radwan, Wang, & Chilakapati, 2008). Glance behavior is one of the most powerful of these measures due to the established relationship between visual demand and crash risk (Liang, Lee, & Yekhshatyan, 2012;Victor et al, 2014).…”

“…Driving simulators have been validated for assessing the demand placed on drivers by smartphones (McWilliams, Ward, Mehler, & Reimer, 2018), performing surrogate research tasks (Wang et al, 2010), driver physiological responses (Reimer & Mehler, 2011), and, to some extent, embedded vehicle HMIs (Klüver et al, 2013, Perez et al, 2013, Santos et al, 2005. Procuction embedded HMIs that have been investigated in simulator validation experiments primarily consist of manual interactions with radios.…”