Speech Pause Patterns in Collaborative Dialogs

Abstract

This chapter discusses the multimodal analysis of human behavioral data from the big data perspective. Though multimodal big data bring tremendous opportunities for related applications, we present current challenges in the domain of multimodal and behavioral analytics. We argue that in the case of analysing human behavior in interaction, we need to shift to the analysis of samples, smaller datasets, before scaling up to large data collections. We describe a dataset developed to study group collaborative interaction from measurable behavioral variables. As a case study, we investigate speech pauses and their patterns in the data, as well as their relationship to the topics of the dialog and the turn-taking mechanism, and we discuss their role in understanding the structure of collaborative interactions as well as in interpreting the behavior of the dialog participants.

A Heuristic Method for Automatic Gaze Detection in Constrained Multi- Modal Dialogue Corpora

“…This work contributes to cognitive infocommunications research [1]- [3], particularly the thread which attends to linguistic and behavioural interaction [4], where there is interest in studying gesture and facial expression alongside linguistic behaviours as recorded in multi-modal data sets, without special purpose eye-trackers or the like [5]- [9]. The field may benefit from using a similar approach to automatic gaze annotation in support of such analyses.…”