Looking for a Deal?

Abstract

Whereas social visual attention has been examined in computer-mediated (e.g., shared screen) or video-mediated (e.g., FaceTime) interaction, it has yet to be studied in mixed-media interfaces that combine video of the conversant along with other UI elements. We analyzed eye gaze of 37 dyads (74 participants) who were tasked with negotiating the price of a new car (as a buyer and seller) using mixed-media video conferencing under competitive or cooperative negotiation instructions (experimental manipulation). We used multidimensional recurrence quantification analysis to extract spatio-temporal patterns corresponding to mutual gaze (individuals look at each other), joint attention (individuals focus on the same elements of the interface), and gaze aversion (an individual looks at their partner, who is looking elsewhere). Our results indicated that joint attention predicted the sum of points attained by the buyer and seller (i.e., the joint score). In contrast, gaze aversion was associated with faster time to complete the negotiation, but with a lower joint score. Unexpectedly, mutual gaze was highly infrequent and unrelated to the negotiation outcomes and none of the gaze patterns predicted subjective perceptions of the negotiation. There were also no effects of gender composition or negotiation condition on the gaze patterns or negotiation outcomes. Our results suggest that social visual attention may operate differently in mixed-media collaborative interfaces than in face-to-face interaction. As mixed-media collaborative interfaces gain prominence, our work can be leveraged to inform the design of gaze-sensitive user interfaces that support remote negotiations among other tasks.

The state of the art of diagnostic multiparty eye tracking in synchronous computer-mediated collaboration

“…Instead of measuring the degree of convergent states between two time series, MdRQA measures the extent of recurring state compositions between numerous temporal data streams (see Figure 5). Thus, MdRQA can be used to investigate more complex processes between individuals by operationalizing constructs of interest based on certain AOI compositions (see, e.g., Vrzakova et al, 2021). The avoidance of eye contact during social interaction, for instance, is not characterized by convergence, but systematic divergence between two participant's gaze positions and therefore could not be computed using CRQA.…”

“…For instance, errors in a presented software code had to be discovered and marked in a programming task (Villamor & Rodrigo, 2018). One of the decision-making tasks required two participants to discuss and agree on eight key features related to a car deal in order to maximize their collective profit (Vrzakova et al, 2021). Experimental tasks associated with CSCL included the collaborative solving of mathematical problems using educational collaboration tools (24%) and the joint creation of concept maps based on previously processed learning materials (12%).…”

“…Two studies additionally operationalized the extent of joint mental effort (8%) by calculating a group's cognitive load based on synchronized pupil size data (Sharma et al, 2021b;Tchanou et al, 2020). Vrzakova et al (2021), on the other hand, also analyzed social gaze dynamics, such as mutual gaze (4%) and gaze aversion (4%). Examined studies investigated correlations between these group-level eye movement metrics and either learning gains (24%) or task-specific performance variables (76%).…”

“…In order to investigate eye movements and gaze patterns of collaborating participants within a corresponding visual space, user interfaces were shared in real time, contained a synchronized area, or were duplicated within dyads (Molinari, 2017;Sharma et al, 2013;Vrzakova et al, 2021). Considering these differences, the degree of coupling differed between tasks.…”

“…Furthermore, gaze behavior has a crucial communicative function (Emery, 2000). For instance, "where one looks, how long, and when" signals engagement, supports rapport building, and regulates turn-taking in natural social interaction (Hessels, 2020;Vrzakova et al, 2021).…”