An Android for Emotional Interaction: Spatiotemporal Validation of Its Facial Expressions

Abstract

Android robots capable of emotional interactions with humans have considerable potential for application to research. While several studies developed androids that can exhibit human-like emotional facial expressions, few have empirically validated androids’ facial expressions. To investigate this issue, we developed an android head called Nikola based on human psychology and conducted three studies to test the validity of its facial expressions. In Study 1, Nikola produced single facial actions, which were evaluated in accordance with the Facial Action Coding System. The results showed that 17 action units were appropriately produced. In Study 2, Nikola produced the prototypical facial expressions for six basic emotions (anger, disgust, fear, happiness, sadness, and surprise), and naïve participants labeled photographs of the expressions. The recognition accuracy of all emotions was higher than chance level. In Study 3, Nikola produced dynamic facial expressions for six basic emotions at four different speeds, and naïve participants evaluated the naturalness of the speed of each expression. The effect of speed differed across emotions, as in previous studies of human expressions. These data validate the spatial and temporal patterns of Nikola’s emotional facial expressions, and suggest that it may be useful for future psychological studies and real-life applications.

Optimizing Facial Expressions of an Android Robot Effectively: a Bayesian Optimization Approach

“…3) BORFEO can generate RFEs that receive higher human ratings for most basic emotions compared with previous work involving the same robot [7]; this finding indicates that BORFEO is a high-performance method. The remainder of this paper is structured as follows.…”

“…1), we have developed, is one of the world's leading androids in this respect. Through a series of human experiments, Sato et al [7] verified that Nikola can produce several basic human-like RFEs.…”

“…Soft and deformable silicone skin is attached to the front of Nikola's skull. Beneath the skin, there are 35 pneumatic actuators with pressure control valves: 29 for facial muscle movement control, 3 for head movement control (roll, pitch, and yaw rotation), and 3 for eyeball control (i.e., panning of the individual eyeballs and tilting of both eyeballs) [7].…”

“…Recent studies have enabled robots to generate facial expressions, either through knowledge-based human coding or in an automated manner [8]. Sato et al [7] used the knowledge-based human coding approach. They identified basic axes for each emotion based on Ekman's theory [9], and then asked a human expert to adjust the parameters of each axis.…”

The spatio-temporal features of perceived-as-genuine and deliberate expressions

“…As Fig 5 shows, with regard to anger, there were more rapid and intense onsets in Components 1 and 2 relative to the perceived-as-genuine expressions. The greater the speed the greater the perceived intensity of anger expressions [17], but rapid speeds are not always understood to be natural as found in recent android research [62]. In line with the accumulated evidence, many scholars have already reported that the temporal aspects of facial expressions are important [63][64][65][66].…”