In a heart beat: Using driver’s physiological changes to determine the quality of a takeover in highly automated vehicles

Abstract

Developing conditionally automated driving systems is on the rise. Vehicles with full longitudinal and latitudinal control will allow drivers to engage in secondary tasks without monitoring the roadway, but users may be required to resume vehicle control to handle critical hazards. The loss of driver's situational awareness increases the potential for accidents. Thus, the automated systems need to estimate the driver's ability to resume control of the driving task. The aim of this study was to assess the physiological behaviour (heart rate and pupil diameter) of drivers. The assessment was performed during two naturalistic secondary tasks. The tasks were the email and the twenty questions task in addition to a control group that did not perform any tasks. The study aimed at finding possible correlations between the driver's physiological data and their responses to a takeover request. A driving simulator study was used to collect data from a total of 33 participants in a repeated measures design to examine the physiological changes during driving and to measure their takeover quality and response time. Secondary tasks induced changes on physiological measures and a small influence on response time. However, there was a strong observed correlation between the physiological measures and response time. Takeover quality in this study was assessed using two new performance measures called PerSpeed and PerAngle. They are identified as the mean percentage change of vehicle's speed and heading angle starting from a takeover request time. Using linear mixed models, there was a strong interaction between task, heart rate and pupil diameter and PerSpeed, PerAngle and response time. This, in turn, provided a measurable understanding of a driver's future responses to the automated system based on the driver's physiological changes to allow better decision making. The present findings of this study emphasised the possibility of building a driver mental state model and prediction system to determine the quality of the driver's responses in a highly automated vehicle. Such results will reduce accidents and enhance the driver's experience in highly automated vehicles.

The Validity of Physiological Measures to Identify Differences in Intrinsic Cognitive Load

“…Heart and respiration measures were recorded in 11 experiments (see Rendon-Velez et al, 2016;Wong and Epps, 2016;Reinerman-Jones et al, 2017;Wu et al, 2017;Lyu et al, 2018;Alrefaie et al, 2019;He et al, 2019;Jaiswal et al, 2019;Ahmad et al, 2020;Digiesi et al, 2020;Gupta et al, 2020;Zakeri et al, 2020). For this category (see Table 1), the most common forms of measures were HR and HRV.…”

“…Combined across the different tasks, the heart measures were capable of finding at least one significant difference in cognitive load in 68% of studies, with exact matches of 64%. Mazur et al, 2016;Rendon-Velez et al, 2016;Wong and Epps, 2016;Hosseini et al, 2017;Yan et al, 2017;Jiménez et al, 2018;Alrefaie et al, 2019;He et al, 2019;Hossain et al, 2019;Vera et al, 2019;Ahmad et al, 2020;Maki-Marttunen et al, 2020;Vanneste et al, 2020;Zakeri et al, 2020). The two most popular measures were pupil diameter and blink rates.…”

The Compatibility between the Takeover Process in Conditional Automated Driving and the Current Geometric Design of the Deceleration Lane in Highway

“…Post-hoc Bonferroni tests showed that a 1 under three Driving Conditions were significantly different with each other, i.e., a 1−5s TOR−time > a 1−8s TOR−time > a 1−manual driving , while no significant main effect of Driving Condition was found on a 2 . Some previous studies reported that hard braking is an indication of the lack of situation awareness, which is always adopted by out-of-loop drivers to acquire longer time for themselves to restore situation awareness [2,3,40]. However, in the mainline near the deceleration lane, braking hard could be a high-risk maneuver which may lead to serious rear-end collisions [42].…”

“…Motor readiness reflected the physical reaction of the driver, while mental readiness reflected the recovery of situation awareness and the ability to make decisions [3,39]. Takeover time was a direct measurement to represent the motor readiness of drivers [40], which was defined as the gap between the onset of the takeover instruction and the moment when the participant resumed the manual control via the steering wheel or the brake pedal. The threshold values of the maneuver were 2 • steering wheel change or 10% brake pedal application [2,5].…”

Legal issues in automated vehicles: critically considering the potential role of consent and interactive digital interfaces

“…Technology factors-safe stop and driver monitoring Designs which allow an AV to default to an emergency manoeuvrer and safe stop, may alleviate the worst cases of drivers being unavailable to take over the controls of a vehicle (Magdici and Althoff, 2016). Systems monitoring heart rate, pupil diameter and gaze may be able to detect when a person is not able to respond to a takeover situation (Alrefaie et al, 2019). These systems could potentially work in conjunction with an emergency fall back of safe stop or safe manoeuvrer, where the driver is not ready (Svensson, 2018).…”