No abstract

In this paper we propose a computational model for the real time generation of nonverbal behaviors supporting the expression of interpersonal attitudes for turn-taking strategies and group formation in multi-party conversations among embodied conversational agents. Starting from the desired attitudes that an agent aims to express towards every other participant, our model produces the nonverbal behavior that should be exhibited in real time to convey such attitudes while managing the group formation and attempting to accomplish the agent's own turn-taking strategy. We also propose an evaluation protocol for similar multi-agent configurations. We conducted a study following this protocol to evaluate our model. Results showed that subjects properly recognized the attitudes expressed by the agents through their nonverbal behavior and turn taking strategies generated by our system.

This paper presents a computational model for managing an Embodied Conversational Agent's first impressions of warmth and competence towards the user. These impressions are important to manage because they can impact users' perception of the agent and their willingness to continue the interaction with the agent. The model aims at detecting user's impression of the agent and producing appropriate agent's verbal and nonverbal behaviours in order to maintain a positive impression of warmth and competence. User's impressions are recognized using a machine learning approach with facial expressions (action units) which are important indicators of users' affective states and intentions. The agent adapts in real-time its verbal and nonverbal behaviour, with a reinforcement learning algorithm that takes user's impressions as reward to select the most appropriate combination of verbal and non-verbal behaviour to perform. A user study to test the model in a contextualized interaction with users is also presented. It was conducted to investigate whether agent's adaptation can positively influence users' impressions of agent's warmth and competence, as well as user's overall perception of the interaction. Our hypotheses were that users' ratings differed when the agents adapted its behaviour according to our reinforcement learning algorithm, compared to when the agent did not adapt its behaviour to user's reactions (i.e., when it randomly selected its behaviours). The study showed a general tendency for the agent to perform better when using our model than in the random condition. Significant results showed that user's ratings about agent's warmth were influenced by their a priori about virtual characters, as well as that users' judged the agent as more competent when it adapted its behaviour compared to random condition.