Language-based multimodal displays for the handover of control in autonomous cars

Abstract

This paper presents a first evaluation of multimodal language-based warnings for handovers of control in autonomous cars. A set of possible handover situations varying in urgency is described. A set of multimodal, language-based warnings for these situations is then introduced. All combinations of audio, tactile and visual warnings for handovers were evaluated in terms of perceived urgency, annoyance and alerting effectiveness. Results showed clear recognition of the warning urgency in this new context, as well as low perceived annoyance overall, and higher perceived effectiveness for critical warnings. The time of transition from self-driving to manual mode in the presence of the warnings was then evaluated. Results showed quicker transitions for highly urgent warnings and poor driving performance for unimodal visual warnings. These results provide a novel set of guidelines for an effective transition of control between car and driver in an autonomous vehicle.

Communicating Awareness and Intent in Autonomous Vehicle-Pedestrian Interaction

“…The major emphasis of related work is on the interaction between the autonomous vehicle and the person inside the vehicle, which we call the driver-centred approach (such as [1,19,21,33]). The pedestrian-centred approach, which focuses on the interaction of the vehicle and the pedestrian, is less explored, and is the focus of our work.…”

Priming Drivers before Handover in Semi-Autonomous Cars

“…Most work has focused on the design of the alert, in terms of the timing and modalities so that it conveys the required urgency [20,32,40,44,54]. For example, Gold et al [20] showed that alerts happening 7 seconds before the incident resulted in more successful take overs compared to alerts that were presented 5 seconds before.…”

“…Indeed, various aspects of handover [20] and in-car alerts [44,54] have received attention in the literature. Currently alerts, such as in the Tesla model S, include a brief alert and immediately handover control to the driver.…”

Am I Driving or Are You or Are We Both? A Taxonomy for Handover and Handback in Automated Driving

“…Most TOR studies start to measure reaction time in the Perception Phase when the system issues the request. For, multimodal notifications, which seem to be most effective (Politis, Brewster, & Pollick, 2015), when does timing start if the stimuli are not simultaneous?…”

“…For that purpose, suggestions about how these phases could be measured (for the most common studies -unscheduled handback situations) are provided in Table 3. Current studies present handover reaction times ranging from in excess of one (Politis, Brewster, & Pollick, 2015) to several seconds, where Radlmayr and Bengler (2015) state transitions of as much as 10 seconds is be suitable for most situations. For studies to be repeatable, measures must be consistently with the 1/5/75 Rule for pedal movements in SAE Recommended Practice J2944 (Society of Automotive Engineers, 2015).…”