Ghost on the Windshield: Employing a Virtual Human Character to Communicate Pedestrian Acknowledgement and Vehicle Intention

Abstract

Pedestrians base their street-crossing decisions on vehicle-centric as well as driver-centric cues. In the future, however, drivers of autonomous vehicles will be preoccupied with non-driving related activities and will thus be unable to provide pedestrians with relevant communicative cues. External human–machine interfaces (eHMIs) hold promise for filling the expected communication gap by providing information about a vehicle’s situational awareness and intention. In this paper, we present an eHMI concept that employs a virtual human character (VHC) to communicate pedestrian acknowledgement and vehicle intention (non-yielding; cruising; yielding). Pedestrian acknowledgement is communicated via gaze direction while vehicle intention is communicated via facial expression. The effectiveness of the proposed anthropomorphic eHMI concept was evaluated in the context of a monitor-based laboratory experiment where the participants performed a crossing intention task (self-paced, two-alternative forced choice) and their accuracy in making appropriate street-crossing decisions was measured. In each trial, they were first presented with a 3D animated sequence of a VHC (male; female) that either looked directly at them or clearly to their right while producing either an emotional (smile; angry expression; surprised expression), a conversational (nod; head shake), or a neutral (neutral expression; cheek puff) facial expression. Then, the participants were asked to imagine they were pedestrians intending to cross a one-way street at a random uncontrolled location when they saw an autonomous vehicle equipped with the eHMI approaching from the right and indicate via mouse click whether they would cross the street in front of the oncoming vehicle or not. An implementation of the proposed concept where non-yielding intention is communicated via the VHC producing either an angry expression, a surprised expression, or a head shake; cruising intention is communicated via the VHC puffing its cheeks; and yielding intention is communicated via the VHC nodding, was shown to be highly effective in ensuring the safety of a single pedestrian or even two co-located pedestrians without compromising traffic flow in either case. The implications for the development of intuitive, culture-transcending eHMIs that can support multiple pedestrians in parallel are discussed.

Smiles and Angry Faces vs. Nods and Head Shakes: Facial Expressions at the Service of Autonomous Vehicles

“…Accordingly, the smile has been utilized to signify yielding intention in various anthropomorphic eHMI concepts on the premise that, in a hypothetical situation where right-of-way is negotiated, a driver could smile at a pedestrian to communicate yielding intention [34,37,41,42,56,62,100]. Having considered the circumstances, the pedestrian would associate friendliness/kindness with being offered safe passage and would, thus, proceed to cross first [101].…”

“…Fittingly, a driver could also decide to nod at a pedestrian to let them know they will be yielded to [1,3,4]. In that case, the pedestrian would associate agreement/cooperativeness with relinquishing right-of-way and would, thus, go ahead and cross first [101]. Indeed, in Rouchitsas and Alm [101], the nod was shown to help pedestrians infer vehicle yielding intention highly effectively in the context of a self-paced crossing intention task, as evidenced by an accuracy of 96.3% in the nod experimental condition.…”

“…A neutral expression, i.e., an alert but blank expression, is a social stimulus that is evaluated as indifferent with respect to valence and does not provide a perceiver with any insight as to what an expresser is feeling, thinking, or planning on doing [97,111,112]. Nonetheless, when employed side by side with a smile in a concept, it is reasonable for the neutral expression to act as a signifier of non-yielding or cruising intention, as it is doubtful that a pedestrian would try crossing the street without having received some confirmation from the driver first [101], and will most likely adopt a look-before-you-leap strategy instead, based on ambiguity aversion [113]. Even so, communicating safety-critical information via an ambiguous expression is inadvisable, as misjudging the intention of an oncoming vehicle may have severe consequences [114,115].…”

“…Even so, communicating safety-critical information via an ambiguous expression is inadvisable, as misjudging the intention of an oncoming vehicle may have severe consequences [114,115]. In fact, Rouchitsas and Alm [101] found the neutral expression to not communicate vehicle cruising intention effectively, as evidenced by an accuracy of only 82.3% for interpretation of vehicle intention, which is lower than the 85% criterion for effectiveness proposed by Kaß et al [116]. Accordingly, more unambiguous expressions for communicating vehicle nonyielding and cruising intention will have to be explored as alternatives if anthropomorphic eHMI concepts are to be utilized safely in the soon-to-be mixed-autonomy traffic.…”

“…Regarding employing emotional expressions to communicate non-yielding intention, Chang [62] evaluated an interface concept where a smile signifies yielding intention and a sad expression signifies non-yielding intention. However, it is highly unlikely that a driver would be saddened by a pedestrian [101], given that sadness is typically felt as a result of loss while a sad facial expression communicates need of help or comfort [97]. Accordingly, it comes as no surprise that said concept was not found to be effective in helping pedestrians decide appropriately whether to cross or not, as evidenced by an accuracy of only 75% for interpretation of vehicle intention.…”