Modeling affective system and episodic memory for social companions

Abstract

Social companions, including virtual humans and social robots, have attracted increasing attention in recent years. The social companions interact with users in a social and humanlike way that makes them believable and trustable. Recent studies have investigated the problem that people tend to feel bored and lose interest rapidly in the first few interactions with social companions. Thus, many studies have started to improve the capabilities of the social companions to preserve user engagement over long-term period. Such companions are required to display consistent personalities in interactions and remember what has happened in the past with particular users. Therefore, our research here focuses on designing the affective system and the episodic memory model, and studies how to integrate them compatibly with social companions to improve user engagement in interactions. Specifically, we have studied (1) how to design a general system architecture for social companions with necessary modules and connections; (2) how to design an affective system to express specific personalities by organizing emotional behaviors in particular patterns, (3) how to design an episodic memory model with more comprehensive similarity measure that can both improve the agent performance in tasks and the user engagement in social companion interactions, and (4) how to construct a benchmark for episodic memory models designed I would like to thank my supervisors Prof. Jianmin Zheng and Prof. Nadia Magnenat-Thalmann, for their time, effort, patience and constant support during this research. Their great knowledge and serious attitude to research work benefit me well. I also want to express my gratitude for Prof. Edie Rasmussen, who is a professor from University of British Columbia and a visiting professor in NTU during the year 2014. She taught me a lot about information retrieval from scratch and discussed with me routinely every week. Without her valuable help and encouragement, I probably couldn't get through the tough time in my PhD. Thank Dr. Budhitama Subagdja to provide me detailed explanation and source codes of the EM-ART model. In addition, I want to give my thank to my colleagues in IMI (Institute for Media Innovation), I have learned quite a lot from them in the four years. Especially, Dr Zerrin Yumak's work provided me a good starting point. I appreciate the financial support for this research provided by SCSE (School of Computer Science and Engineering) and IMI, and the space and equipment provided by Institute for Media Innovation (IMI), Nanyang Technological University. Deep thanks to my family and all my friends. In particular, I thank my parents for their encouragement and listening and my wife Panpan Cai for her support and love.