Dynamic Bayesian Network Modelling of User Affect and Perceptions of a Teleoperated Robot Coach during Longitudinal Mindfulness Training

Abstract

Longitudinal interaction studies with Socially Assistive Robots (SARs) are crucial to ensure that the robot is relevant for long-term use and its perceptions are not prone to the novelty effect. In this paper, we present a dynamic Bayesian network (DBN) model to capture the longitudinal interactions participants had with a teleoperated robot coach (RC) delivering mindfulness training sessions. The DBN model is used to study complex, temporal interactions between the participants' self-reported personality traits, weekly baseline wellbeing scores, session ratings, and participants' facial AUs elicited during the sessions during a 5-week longitudinal study. DBN modelling involves learning a graphical representation that facilitates intuitive understanding of how multiple components contribute to the longitudinal changes in session ratings corresponding to the perceptions of the robot coach and the changes in the relaxation/calm levels before and after each session. The learnt model captures the following within-and between-sessions aspects of the longitudinal interaction study -(i) influences of the 5 personality dimensions on the facial AU states and the session ratings; (ii) influence of facial AU states on the session ratings; and (iii) the influences within the items of the session ratings. The structure of the DBN was learnt using the first 3 time-points and the obtained model is used to predict the session ratings of the remaining 2 time-points of the 5-week longitudinal data. The goodness-of-fit of the model is quantified in terms of subject-wise RM SE and R 2 scores. We also demonstrate two applications of the model, namely imputation of missing values in the dataset and estimation of longitudinal session ratings of a new participant with a given personality profile. The obtained DBN model thus facilitates learning of conditional dependency structure between variables in the longitudinal data and offers inferences and conceptual understanding which is not possible through other regression methodologies.