Measuring Collaboration Load With Pupillary Responses - Implications for the Design of Instructions in Task-Oriented HRI

Abstract

In face-to-face interaction, speakers establish common ground incrementally, the mutual belief of understanding. Instead of constructing “one-shot” complete utterances, speakers tend to package pieces of information in smaller fragments (what Clark calls “installments”). The aim of this paper was to investigate how speakers' fragmented construction of utterances affect the cognitive load of the conversational partners during utterance production and comprehension. In a collaborative furniture assembly, participants instructed each other how to build an IKEA stool. Pupil diameter was measured as an outcome of effort and cognitive processing in the collaborative task. Pupillometry data and eye-gaze behaviour indicated that more cognitive resources were required by speakers to construct fragmented rather than non-fragmented utterances. Such construction of utterances by audience design was associated with higher cognitive load for speakers. We also found that listeners' cognitive resources were decreased in each new speaker utterance, suggesting that speakers' efforts in the fragmented construction of utterances were successful to resolve ambiguities. The results indicated that speaking in fragments is beneficial for minimising collaboration load, however, adapting to listeners is a demanding task. We discuss implications for future empirical research on the design of task-oriented human-robot interactions, and how assistive social robots may benefit from the production of fragmented instructions.

Virtual inverted classroom to replace in-person radiology lectures at the time of the COVID-19 pandemic - a prospective evaluation and historic comparison

“…Therefore, depending on emerging uncertainty of listeners, speakers tend to fragment their information. Although this strategy is cognitively demanding for the speaker, it avoids additional collaborative effort [ 26 ]. As uncertainty is mostly expressed non-verbally [ 18 ], with virtual conversation, the level of understanding is difficult to recognize and thus, conversational behaviour pattern might be disturbed.…”

How do enhanced videos support generative learning and conceptual understanding in individuals and groups?

“…Forth, concerning learning processes, we suggest addressing the close relationship between micro-and task-actions in future research, for example by conducting behavior sequence analyses (see for example, Sinha et al, 2014). Last, cognitive load should be considered by using measurements for intrinsic, extraneous, and germane load with validated instruments (e.g., Klepsch et al, 2017) and by focusing on collaborative load (e.g., Kontogiorgos & Gustafson, 2021).…”