A pupil response system using hemispherical displays for enhancing affective conveyance

Abstract

In human interaction and communication, not only verbal messages but also nonverbal behaviors such as facial expressions, body movements, gazes and pupil responses play an important role in expressions of talker's affect. These expressions encourage to read the emotional cues and to cause the sharing of embodiment and empathy. We focused on the pupil response which is closely related to human affect, and developed an embodied communication system in which an interactive CG character generates the pupil response as well as communicative actions and movements such as nodding and body movements by speech input. In addition, it was confirmed that the pupil response is effective for supporting the embodied interaction and communication using the developed system. In this paper, in order to realize the smooth interaction between human and robot, we developed a pupil response system using hemispherical displays for enhancing affective conveyance. This system looks like robot's eyeballs and expresses vivid pupil response by speech input. We carried out a sensory evaluation experiment under the condition that the developed system speaks. The results demonstrated that the system effectively enhances affective conveyance.

A speech-driven pupil response system with affective highlight by virtual lighting

“…The appearance of the pupil response interface that has been developed so far is shown in Fig. 1 (Sejima et al, 2019). The pupil response interface consists of a notebook PC equipped with Windows7 (CPU: Corei-7 1.8GHz, Memory: 8GB, Graphics: Intel (R) UHD Graphics), hemispherical display (Gakken WORLDEYE), video distributor (SANWA 400-VGA003 TSL).…”

“…We have already developed the pupil response interface that is eye-only in appearance (Sejima et al, 2019). This pupil response interface mimics the eyeball by projecting the 3DCG model in virtual space onto a hemispherical display (Fig.…”

“…In addition, it is possible to generate enlargement / reduction of the pupil dynamically by controlling the 3DCG model in conjunction with speech input. We evaluated this pupil response interface and demonstrated that the interest and enthusiasm of the social robot itself can be expressed by dilating the pupil in synchronization with the talker's speech (Sejima et al, 2019). The developed pupil response interface can generate various affective expressions by controlling the 3DCG model in virtual space.…”