Feedback interpretation based on facial expressions in human-robot interaction

Abstract

Abstract-In everyday conversation besides speech people also communicate by means of nonverbal cues. Facial expressions are one important cue, as they can provide useful information about the conversation, for instance, whether the interlocutor seems to understand or appears to be puzzled. Similarly, in human-robot interaction facial expressions also give feedback about the interaction situation.We present a Wizard of Oz user study in an object-teaching scenario where subjects showed several objects to a robot and taught the objects' names. Afterward, the robot should term the objects correctly. In a first evaluation, we let other people watch short video sequences of this study. They decided by looking at the face of the human whether the answer of the robot was correct (unproblematic situation) or incorrect (problematic situation). We conducted the experiments under specific conditions by varying the amount of temporal and visual context information and compare the results with related experiments described in the literature.

Enabling robots to make use of the structure of human actions - A user study employing Acoustic Packaging

“…The order of the trials was counterbalanced. Based on findings from previous research [3], all other utterances were distributed equally in all scripts ("Pardon?" twice, "I don't know the word" once, etc.).…”

Facial expressions as feedback cue in human-robot interaction—a comparison between human and automatic recognition performances

“…The video database used in this paper is the object teaching corpus presented by Lang et al [12]. It contains videos of people interacting with the robot "Biron" [11] in an object-teaching scenario.…”

“…On the other hand, the classification problem is expected to be hard, as the average human performance is only 82% [12] (please see section 5). For the subsequent investigations, we used only the best performing variant of each feature (marked in bold in table 1), except for the gabor energy filters, where we used variant "gab-4" instead of "gab-8" because of the lower feature vector dimensionality (640 compared to 2,560) and the only marginal difference in the classification rate (0.3% means just one more video classified correctly).…”

“…According to experiences in this field (as reported by Caridakis et al [3] and Lang et al [12], for instance), most of these emotional expressions occur much less frequently in human-robot interaction than facial expressions that carry some communicative semantics. Some examples of this kind of "communicative" facial expressions are looking disappointed or puzzled, appearing to agree or disagree with the interlocutor, or seeming satisfied with or frustrated by the situation.…”

“…In this paper, we investigate the automatic recognition by means of standard pattern recognition techniques of a special type of communicative facial expressions: the recognition of valence in terms of success and failure, following the approach of Lang et al [12]. In this context, success means that a particular interaction with the robot could be performed as desired, whereas failure means that some problem occured.…”