Modeling Team-level Multimodal Dynamics during Multiparty Collaboration

Eloy, Lucca; Stewart, Angela; Amon, Mary Jean; Reinhardt, Caroline; Michaels, Amanda; Sun, Chen; Shute, Valerie J.; Duran, Nicholas D.; D’Mello, Sidney K.

doi:10.1145/3340555.3353748

Cited by 28 publications

(31 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multidimensional quantification analysis (MdRQA) [117] is another type of RQA that accommodates multiple signals, (e.g., from multiple participants [113] and/or from multiple modalities (gaze, speech, activity on the screen) [1,30,35]. Notably, MdRQA contrasts CRQA by measuring the extent to which the multidimensional system visits repeated (recurrent) states, irrespective of whether or not the individual signals are in alignment.…”

Section: Principles Of Mdrqamentioning

confidence: 99%

Looking for a Deal?

Vrzáková

Amon

Rees

et al. 2021

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

Whereas social visual attention has been examined in computer-mediated (e.g., shared screen) or video-mediated (e.g., FaceTime) interaction, it has yet to be studied in mixed-media interfaces that combine video of the conversant along with other UI elements. We analyzed eye gaze of 37 dyads (74 participants) who were tasked with negotiating the price of a new car (as a buyer and seller) using mixed-media video conferencing under competitive or cooperative negotiation instructions (experimental manipulation). We used multidimensional recurrence quantification analysis to extract spatio-temporal patterns corresponding to mutual gaze (individuals look at each other), joint attention (individuals focus on the same elements of the interface), and gaze aversion (an individual looks at their partner, who is looking elsewhere). Our results indicated that joint attention predicted the sum of points attained by the buyer and seller (i.e., the joint score). In contrast, gaze aversion was associated with faster time to complete the negotiation, but with a lower joint score. Unexpectedly, mutual gaze was highly infrequent and unrelated to the negotiation outcomes and none of the gaze patterns predicted subjective perceptions of the negotiation. There were also no effects of gender composition or negotiation condition on the gaze patterns or negotiation outcomes. Our results suggest that social visual attention may operate differently in mixed-media collaborative interfaces than in face-to-face interaction. As mixed-media collaborative interfaces gain prominence, our work can be leveraged to inform the design of gaze-sensitive user interfaces that support remote negotiations among other tasks.

show abstract

Section: Principles Of Mdrqamentioning

confidence: 99%

Looking for a Deal?

Vrzáková

Amon

Rees

et al. 2021

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Verbal contents include processing of audio transcripts to extract speech and linguistic features. This has been achieved using latent semantic analysis (LSA) [52], IBM speech to text services [68,38] and Linguistic Inquiry and Word Count text analysis program [1]. Nonverbal content based measurements can include information about number of communication attempts such as button-clicks and acknowledgements to attempted communication [69], by extracting vocal tone frequencies [25,30] and team interaction instances of attempted problem-solutions, general meetings, debriefing and brainstorming [37].…”

Section: B Behavioural Observationsmentioning

confidence: 99%

“…Finally, team members actions has been analysed using accelerometers [48] or video recordings [72,45,66]. Physical bodily movements such as postural sway, hand coordination and face and head movements are also used to evaluate joinactivity in relation to performance [37,38,68,45].…”

Section: B Behavioural Observationsmentioning

confidence: 99%

“…CRQA has been employed to observe the relationships such as joint activity and coordination between the team members [48,67,53,49,63]. Other versions of CRQA are multidimensional-RQA (MdRQA) [48,38] also referred to as multivariate-RQA (MvRQA) [76] offer similar measurements but with an advantage of comparing multiple signals originating from two team individuals.…”

Section: A Compliance Metricsmentioning

confidence: 99%

“…Such an experimental setup is necessary to test and evaluate team related theoretical hypotheses [35]. In recent years, team research has witnessed a steady rise in the number of studies experimenting with combination of PBRs (also referred to as multimodal techniques) [36,37,38]. Observing this trend, this review presents insights from team studies employing multimodal PBR approach.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fundamental Requirements for Contemporary Performance Evaluation of Emergency-Response and Safety-Critical Teams

Veerabhadrappa¹

2021

Preprint

View full text Add to dashboard Cite

This is a review-article which selectively reviews key concepts adopted in team performance evaluation literature. This review intends to promote future innovative methods for objective quantification and analysis of team performance. The review summarizes methods, experimental frameworks, sensors for physiological and behavioral recordings, data processing to derive team level objective measures, in the light of team performance evaluation. Observing the advancements in sensor technologies and computation power, towards advancing team performance evaluation, this review summarizes some of the current multimodal based team research. Finally, the review provides suggestions on aspects that the future research focus to overcome some of existing limitations and drawbacks. <br>

show abstract