Speech-driven embodied entrainment character system with pupillary response

A pupil response system using hemispherical displays for enhancing affective conveyance

“…According to the previous report in which pupil size was investigated, pupil dilation brought favorable impressions (Hess, 1965;Oosaka et al, 2002). In addition, our previous study demonstrated that the enhanced pupil response of talker's avatar improves the impressions of vividness, enthusiasm, and interest level in avatar-mediated communication (Sejima et al, 2016). In this experiment, the response mode (B) in which the pupil reacts to the speech was highly evaluated to be favorable to that of the static mode (A) in all items.…”

“…The analysis demonstrated that the speaker's pupil was clearly dilated in both face-to-face and non-face-to-face communications. Based on the results, we developed an advanced embodied communication system in which an interactive CG character generates the pupil response as well as communicative actions and movements such as nodding and body movements by speech input (Sejima et al, 2016). In addition, it was confirmed that the pupil response is effective for supporting affective conveyance between talkers using the developed system.…”

“…Furthermore, it was reported that the line-of-sight is important for enhancing embodied interaction in human-robot communication (Kanda et al, 2003). In order to develop an advanced communication robot that enhances affective conveyance with line-of-sight, it is effective to express the pupil response that enhances own affect as well as the gaze and eye-contact (Sejima et al, 2016).…”

“…In addition, the pupil response plays an important role in expressions of affect (Hess, 1965). Focusing on the pupil response in human communication, we analyzed the pupil response during human face-to-face communication and non-face-to-face communication (Sejima et al, 2016). The analysis demonstrated that the speaker's pupil was clearly dilated in both face-to-face and non-face-to-face communications.…”

A video communication system with a virtual pupil CG superimposed on the partner’s pupil

“…Focusing on the synchrony of embodied rhythms in human interaction and communication, we developed an interactive CG character called "InterActor" that generates pupil responses which are closely related to human affect as well as communicative actions and movements such as nodding and body movements based on the burst-pause of utterance (Sejima et al, 2016). In addition, we carried out the communication experiment, and demonstrated that not only body Sejima, Maeda, Sato and Watanabe, Journal of Advanced Mechanical Design, Systems, and Manufacturing, Vol.14, No.6 (2020) movements but also the pupil responses were effective for enhancing affective conveyance and empathy by being synchronized with the timing of response (Sejima et al, 2016). Therefore, it is expected that an active video communication wherein talkers can deepen embodied interaction is provided by visualization of the pupil response.…”