Toward Alignment with a Virtual Human - Achieving Joint Attention

Abstract

Abstract. Alignment forms the basis of successful communication. It can be seen as the most efficient means for action coordination of cooperating agents, covering adaptation processes operating without an explicit exchange of information states. One critical condition of alignment consists of joint attention. We present work on equipping a virtual human with the capability of reaching joint attention with its human interlocutor. On the one hand, mechanisms to detect the human's focus of attention are employed. On the other hand, basic cognitive as well as intentional processes underlying the phenomenon of joint attention are incorporated in our agent's cognitive architecture. In this context, a dynamic working memory and a partner model accounting for theory of mind and intentionality are crucial constituents.

Formalizing Joint Attention in Cooperative Interaction with a Virtual Human

“…[10]: The agent needs to (r1) track the attentional behavior of the other by gaze monitoring and (r2) derive candidate objects the interlocutor may be focusing on. Further, the agent has to (r3) infer whether attentional direction cues of the interlocutor are uttered intentionally.…”

“…Our CASEC architecture (Cognitive Architecture for a Situated Embodied Cooperator) [10] adopts the BDI paradigm of rational agents [13] applying modal logic as a specification language, but additionally integrates a dynamic working memory. The formalism used to specify goals and beliefs builds on the possible worlds approach.…”

“…According to Pickering and Garrod [9] successful communication is based on joint processes, called alignment, which realize action coordination between interlocutors without an explicit exchange of information states. In previous work we have argued [10] that one central condition of such alignment processes consists of joint attention and that activation of a dynamic working memory and a partner model are crucial constituents.…”

“…The human interlocutor's focus of attention is inferred from her overt behaviors, and focused objects are activated as salient in the agent's dynamic working memory; for detail cf. [10].…”

“…This model provides a theoretical framework for a cooperative interaction scenario with the virtual human Max and the CASEC cognitive architecture introduced in [10]. In Section 2, we firstly introduce the use of activation values in modal logic and derive a definition of attention in Section 3.…”