Causal Inference in Nonverbal Dyadic Communication with Relevant Interval Selection and Granger Causality

Abstract

Human nonverbal emotional communication in dyadic dialogs is a process of mutual influence and adaptation. Identifying the direction of influence, or cause-effect relation between participants, is a challenging task due to two main obstacles. First, distinct emotions might not be clearly visible. Second, participants causeeffect relation is transient and variant over time. In this paper, we address these difficulties by using facial expressions that can be present even when strong distinct facial emotions are not visible. We also propose to apply a relevant interval selection approach prior to causal inference to identify those transient intervals where adaptation process occurs. To identify the direction of influence, we apply the concept of Granger causality to the time series of facial expressions on the set of relevant intervals. We tested our approach on synthetic data and then applied it to newly, experimentally obtained data. Here, we were able to show that a more sensitive facial expression detection algorithm and a relevant interval detection approach is most promising to reveal the cause-effect pattern for dyadic communication in various instructed interaction conditions. arXiv:1810.12171v1 [cs.CV] 29 Oct 2018 4. We propose a method for the selection of the relevant time intervals where GC should be applied, and show based on synthetic as well as real data, the superiority of the proposed method in detecting cause-effect relations when compared to applying GC on the full time series.

Analysing the Direction of Emotional Influence in Nonverbal Dyadic Communication: A Facial-Expression Study

“…For feature extraction, we used OpenFace 2.0 [29], [40] which is a state-of-the-art, open-source tool for landmark detection; it estimates AUs based on landmark positions. OpenFace preserves much of the information by regressing AUs instead of only classifying them and is capable of extracting 17 different AUs (1,2,4,5,6,7,9,10,12,14,15,17,20,23,25,26,45) with an intensity scaled from 0 to 5. Figure 4 illustrates the detection of landmarks and AUs for an example image.…”

“…We present a relevant interval selection approach that we use prior to causal inference to identify those transient intervals where causal inference should be applied. The validation of the relevant interval selection on synthetic data for improved causal inference has been presented in [20] along with initial results of this study on small data set. Here we show based on a larger real data set the superiority of such an approach in detecting the direction of emotional influence when compared to applying GC test on the entire time-series.…”