An experimental approach to study the physiology of natural social interactions

Abstract

The classical experimental methodology is ill-suited for the investigation of the behavioral and physiological correlates of natural social interactions. A new experimental approach combining a natural conversation between two persons with control conditions is proposed in this paper. Behavior, including gaze direction and speech, and physiology, including electrodermal activity, are recorded during a discussion between two participants through videoconferencing. Control for the social aspect of the interaction is provided by the use of an artificial agent and of videoed conditions. A cover story provides spurious explanations for the purpose of the experiment and for the recordings, as well as a controlled and engaging topic of discussion. Preprocessing entails transforming raw measurements into boxcar and delta functions time series indicating when a certain behaviour or physiological event is present. The preliminary analysis presented here consists in finding statistically significant differences between experimental conditions in the temporal associations between behavioral and physiological time series. Significant results validate the experimental approach and further developments including more elaborate analysis and adaptation of the paradigm to functional MRI are discussed.

Brain activity during reciprocal social interaction investigated using conversational robots as control condition

“…A recent behavioural study comparing human-human with human-robot conversations [25] was adapted to the fMRI environment. The experimental factor was the nature of the INTERACTING AGENT (HUMAN versus ROBOT), in a within-subject, block-design.…”

“…Furhat OS allowed us to control its responses through a Wizard of Oz (WOZ): unbeknown to the participant, the confederate was controlling the robot remotely. The robot conversational feedbacks were largely based on actual human interactions recorded during the previous behavioural study [25]. A WOZ user interface was created with Furhat OS displaying buttons on a web browser running on a tablet allowing the human controller to launch pre-programmed conversational feedback.…”

Grammars of action in human behavior and evolution

“…Artificial tasks designed to isolate such elements of stone tool-making have provided insight into the demands of specific behavioral components 42,43 , but this approach sacrifices ecological validity and cannot address real-world behaviors that necessarily differ on multiple dimensions at the same time. For example, similar challenges confront attempts to study language as it is actually used by people to communicate rather than in the form of controlled laboratory manipulations [44][45][46] . In the current case, it is not possible to control lowerlevel kinematic, geometric, and visual features of stone tool-making without also altering the higher-level action structure that emerges from them and seriously compromising the basis for analogy with real, archaeologically-documented behaviors.…”

Lilia, A Showcase for Fast Bootstrap of Conversation-Like Dialogues Based on a Goal-Oriented System

“…The experiments presented in the paper concern a chit-chat dialogue system framed in a goal-oriented dialogue task. In this context, users discuss with the system about an image (out of a small predefined set of 6), and they tried jointly to discover the message conveyed by the image, as described in [5]. The discussion is expected to last only one minute at most.…”