Dynamic perception-production oscillation model in human-machine communication

Abstract

The goal of the present article is to introduce a new concept of a perception-production timing model in human-machine communication. The model implements a low-level cognitive timing and coordination mechanism. The basic element of the model is a dynamic oscillator capable of tracking reoccurring events in time. The organization of the oscillators in a network is being referred to as the Dynamic PerceptionProduction Oscillation Model (DPPOM). The DPPOM is largely based on findings in psychological and phonetic experiments on timing in speech perception and production. It consists of two sub-systems, a perception sub-system and a production sub-system. The perception sub-system accounts for information clustering in an input sequence of events. The production sub-system accounts for speech production rhythmically entrained to the input sequence. We propose a system architecture integrating both sub-systems, providing a flexible mechanism for perception-production timing in dialogues. The model's functionality was evaluated in two experiments.

An End-to-End Conversational Style Matching Agent

“…Conversational style matching can be seen as a type of entrainment, as it relates to people synchronizing their behavior within an interaction [2,15]. Entrainment has been used in related work to generate more accurate and realistic reactions [20] or backchannels [18]. Levitan et al implemented acoustic-prosodic entrainment in a conversational avatar and observed an increase in perceived reliability and likability of the system [22].…”

Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent

“…Along these lines, an ECA has already been presented that moves according to the rhythm of music as processed by entraining oscillators [17]. Models that involve entrainment are useful in that they can be helpful in the anticipation of turn ends or temporal windows, for example for backchannels, while dynamically adapting to speech tempo by period adaptation [16].…”