Inferring Interactivity From Gaze Patterns During Triadic Person-Object-Agent Interactions

Abstract

Observing others’ gaze informs us about relevant matters in the environment. Humans’ sensitivity to gaze cues and our ability to use this information to focus our own attention is crucial to learning, social coordination, and survival. Gaze can also be a deliberate social signal which captures and directs the gaze of others toward an object of interest. In the current study, we investigated whether the intention to actively communicate one’s own attentional focus can be inferred from the dynamics of gaze alone. We used a triadic gaze interaction paradigm based on the recently proposed classification of attentional states and respective gaze patterns in person-object-person interactions, the so-called “social gaze space (SGS).” Twenty-eight participants interacted with a computer controlled virtual agent while they assumed to interact with a real human. During the experiment, the virtual agent engaged in various gaze patterns which were determined by the agent’s attentional communicative state, as described by the concept of SGS. After each interaction, participants were asked to judge whether the other person was trying to deliberately interact with them. Results show that participants were able to infer the communicative intention solely from the agent’s gaze behavior. The results substantiate claims about the pivotal role of gaze in social coordination and relationship formation. Our results further reveal that social expectations are reflected in differential responses to the displayed gaze patterns and may be crucial for impression formation during gaze-based interaction. To the best of our knowledge, this is the first study to document the experience of interactivity in continuous and contingent triadic gaze interactions.

Impaired recognition of interactive intentions in adults with autism spectrum disorder not attributable to differences in visual attention or coordination via eye contact and joint attention

“…The diagnosis then had to be confirmed in two independent and extensive clinical interviews by two separate professional clinicians according to ICD-10 criteria 2 . After the exclusion of 5 participants (due to missing data or mistrust of the cover story) the remaining 21 subjects (6 identifying as female, 15 as male; aged 22–54, mean = 40.86, SD = 10.36) were compared to a group of 24 control subjects (with an overlap to the population reported in 61 ), without any record of psychiatric or neurological illnesses (10 identifying as female, 14 as male; aged 23–58, mean = 39.00, SD = 12.76). Demographic data and the Autism-Spectrum-Quotient (AQ; Baron-Cohen et al, 2001b) were obtained from all subjects.…”

Temporal Behavioral Parameters of On-Going Gaze Encounters in a Virtual Environment

“…For example, we recently applied this toolbox to study the inference of communicative intent from gaze cues. To this end, we systematically manipulated the interactive states of an agent to determine differential gaze patterns of participants and their impact on the perception of deliberate communicative intent (Jording et al, 2019). Our virtual agent tool also has a high potential to be used for more complex scenarios beyond the "Social Gaze Space".…”

“…Advances in the technical development and computational power sparked the emergence of social, algorithmically controlled agents for face-to-face interactions. Such agents are increasingly used in diverse contexts, for example as assistants for "customer relations" (e.g., Kopp et al, 2005;Heaven, 2018), in interactive teaching contexts (e.g., Lee et al, 2015;Mabanza, 2016), andbasic scientific research (e.g., von der Pütten et al, 2010;Courgeon et al, 2014;Grynszpan et al, 2017;Jording et al, 2019); for a general review on social robots see Mavridis (2015) and for more examples see Hoekstra et al (2007), Pfeiffer et al (2011), Gratch et al (2013), Courgeon andClavel (2013), andPelachaud (2015). Compared to these approaches, our framework is focused on the conceptual framework of Social Gaze States (Jording et al, 2018) which provides a situation-specific taxonomy and exhaustive description of a specific behavior of interest.…”

Theory of mind and decision science: Towards a typology of tasks and computational models

“…Interactivity (Byom and Mutlu, 2013;Jording et al, 2019) in our proposed typology of social cognition tasks refers to a combination of the social distance or face-to-face context (e.g., still photos, recorded video, live video, interactive live video, interactive in person; Spezio, Huang, Castelli, & Adolphs, 2007), the personal relevance, the task-dependent consequences of a social cognition task (Bublatzky et al, 2017), and the level of involvement of multiple agents (Norris et al, 2014). Interactivity is a dimension of socially oriented tasks that ranges from purely passive spectatorial observation to full consequential interaction.…”