Analysis of Respiration for Prediction of "Who Will Be Next Speaker and When?" in Multi-Party Meetings

Abstract

To build a model for predicting the next speaker and the start time of the next utterance in multi-party meetings, we performed a fundamental study of how respiration could be effective for the prediction model. The results of the analysis reveal that a speaker inhales more rapidly and quickly right after the end of a unit of utterance in turn-keeping. The next speaker takes a bigger breath toward speaking in turn-changing than listeners who will not become the next speaker. Based on the results of the analysis, we constructed the prediction models to evaluate how effective the parameters are. The results of the evaluation suggest that the speaker's inhalation right after a unit of utterance, such as the start time from the end of the unit of utterance and the slope and duration of the inhalation phase, is effective for predicting whether turn-keeping or turn-changing happen about 350 ms before the start time of the next utterance on average and that listener's inhalation before the next utterance, such as the maximal inspiration and amplitude of the inhalation phase, is effective for predicting the next speaker in turn-changing about 900 ms before the start time of the next utterance on average.

Respiratory Turn-Taking Cues

“…inhalation duration and inhalation delay. Thus, previous findings of temporal compression in turn-keeping [4,5,17] could be replicated. Contrary to previous findings [17], we have shown that inhalation amplitude and inhalation starting level also make significant contributions to predictions of turn-taking behaviour.…”

“…We arrived at these models by hierarchical entry of predictors [16]. We first entered inhalation duration as this was one of the two most robust features in [6], and this improved all three models significantly compared to a model where only the constant was included. We entered inhalation amplitude next, and this also improved all models significantly.…”

“…The topic has been addressed most comprehensively by Ishii et al [6] using recordings of Japanese four-party conversations on a pre-assigned topic. The authors found significant differences between turn-taking and turn-keeping.…”

Respiratory Belts and Whistles: A Preliminary Study of Breathing Acoustics for Turn-Taking

“…So far, however, there has been little work on whether breathing sounds are used as cues to turnmanagement. Admittedly, several authors, including the present ones, have suggested that breathing is a turn-taking cue [20,21,22,23,24], however the evidence was based on breathing kinematics measured with elastic belts wrapped around speakers' thorax. While this method delivers a gold standard in the detection of respiratory events [12], it is relatively invasive and difficult to incorporate into a spoken dialogue system, not least because it requires a relatively time-consuming calibration procedure.…”

Overlapping speech, utterance duration and affective content in HHI and HCI - An comparison

“…One important requirement for a fluent and successful conversation is an efficient turn-taking, which has to be organized by specific "underlying mechanisms", such as intonation, semantic cues, facial expressions, eye contact, breathing, and gestures [2], [3], [4]. In the organization of turn-taking and to evaluate the conversation, overlapping speech has a major role.…”