Making Them Remember—Emotional Virtual Characters with Memory

Abstract

Research on virtual characters has been ongoing for the past 20 years. Early efforts focused mostly on making the characters move and speak-that is, on body and facial animation. Simultaneously, researchers worked on making characters look convincing by adding animation and rendering hair, clothes, and muscles. The next step was to increase artists' interactive control over characters so that it was easier to create convincing video games and cinema. Today, research into user interactivity has come to the forefront. It's no longer sufficient for characters to simply look like imitations of humans. They must behave like humans, too. This fact drives research into emotional and conversational virtual characters, or embodied conversational agents. The goal is to create a virtual character that has a human-like personality and that can emotionally respond while conversing with a user. To this end, some researchers mathematically model emotions, behavior, mood, and personality for virtual characters. As we describe here, researchers can use these models to create an emotionally responsive character. However, such models lack the critical component of memory-a memory of not just events but also past emotional interaction.We've developed a memory-based emotion model that uses the memory of past interactions to build long-term relationships between the virtual character and users. We combine this model with stateof-the-art animation blending to generate smooth animation for the character during the interaction. To make the interaction more natural, we also use face recognition techniques; the character can thus "remember" a user's face and automatically adjust the current interaction on the basis of its existing relationship with the user. Finally, to increase the user's immersion, we place a life-sized character in a real environment using marker-based augmented reality (AR) techniques. Our example application is Eva, a geography teacher who has multiple interactions with two student users. Modeling Realistic CharactersTo create realistic characters, we must create models based on three general aspects: emotion, mood and personality, and relationship. Modeling EmotionsEmotions have proven effects on cognitive processes such as action selection, learning, memory, motivation, and planning. Our emotions both motivate our decisions and have impact on our actions. As such, they're a key mechanism for controlling virtual-character behavior by both creating characters' personality and automatically producing animations by simulating characters' internal dynamics.Jonathan Gratch and Stacy Marsella define two methods for modeling emotion in lifelike characters: communicative-driven methods and simulation-based methods.1 Communicative-driven methods treat emotional displays as a means of communication. These systems don't internally calculate emotion; instead, they select an emoThe search for the perfect virtual character is on, but the moment users interact with characters, any illusion that we've found it is broken. Adding mem...

Virtual Assistive Companions for Older Adults: Qualitative Field Study and Design Implications

“…In order to motivate and engage the older adults to remain active, the agent should recognize the user's emotions, analyse their behaviour and deliver appropriately tailored empathetic feedback, in a similar way a real-life human companion would do. This will be achieved during the project CaMeLi [10] by using existing and established technologies for emotion recognition [29], dialogue management [21], virtual human simulation [17] etc. These technologies will be adapted to fulfill the user requirements and similarly to previous studies [21] well-defined dialogues for the scenario should increase the accuracy and the fluency of the interaction.…”

“…Additionally, to work towards a relationship of trust with its user, the companion should be designed to gather information during the interaction, for example by asking questions, and to remember past events and conversations. Researchers have explored the use of virtual agents able to create episodic memories [17] making them able to respond more adequately and adapt to the user's specific needs across time. The CaMeLi virtual companion will build upon the foundations of this research.…”

Toward a Computational Model of Social Relations for Artificial Companions

“…The agent can thus refer to previous interactions with the same user, providing customized advices. In Eva [15], the behavior is calculated according to its personality, mood, emotions and social relations toward the user. The relation is modeled according to two axes of dominance and appreciation, and evolves according to its emotions.…”

WHO ARE YOU? - On the Acquisition of Information about People for an Agent that Remembers

“…Agents equipped with autobiographical memory, e.g. (Kasap et al, 2009), or episodic memory, cf. (Brom et al, 2007), are enabled to remember their own experiences.…”

“…Agents equipped with autobiographic or episodic memories are able to remember past events they experienced either in a virtual world on their own (Brom et al, 2007), or with people they interacted with (Kasap et al, 2009). The agent described in the latter, e.g.…”