Quantification of nonverbal synchrony using linear time series analysis methods: Lack of convergent validity and evidence for facets of synchrony

Abstract

Nonverbal synchrony describes coordination of the nonverbal behavior of two interacting partners. Additionally, it seems to be important in human interactions, such as during psychotherapy. Currently, there are several options for the automated determination of synchrony based on linear time series analysis methods (TSAMs). However, investigations into whether the different methods measure the same construct have been missing. In this study, N = 84 patient-therapist dyads were videotaped during psychotherapy sessions. Motion energy analysis was used to assess body movements. We applied seven different TSAMs and recorded multiple output scores (average synchrony, maximum synchrony, and frequency of synchrony; in total, N = 16 scores). Convergent validity was examined using correlations of the output scores and exploratory factor analysis. Additionally, two criterion-based validations were conducted: investigations of concordant validity with a more generalized nonlinear method, and of the predictive validity of the synchrony scores for improvement in interpersonal problems at the end of therapy. We found that the synchrony measures only partially correlated with each other. The factor analysis did not support a common-factor model. A three-factor model with a second-order synchrony variable showed the best fit for eight of the selected synchrony scores. Only some synchrony scores were able to predict improvement at the end of therapy. We concluded that the considered TSAMs do not measure the same synchrony construct, but different facets of synchrony: the strength of synchrony of the total interaction, the strength of synchrony during synchronization intervals, and the frequency of synchrony. Keywords Nonverbal behavior. Movement synchrony. Motion energy analysis. Time series analysis. Convergent validity Currently, body movements can be assessed fully automatically and with high time resolution (e.g., 25 times per second) using either motion-tracking, motion capture devices, or video-based algorithms (Delaherche et al., 2012). Motion energy analysis (MEA) is a method that quantifies the intensity of videotaped movements frame-wise (Grammer, Honda, Juette, & Schmitt, 1999). By determining a region of interest (ROI) for each of two videotaped individuals (e.g., a patient and therapist during a psychotherapy session), two time series can be generated displaying the time course of the individuals' body movements. This technique has several advantages: (1) it is less timeconsuming than collecting human ratings; (2) it is highly objective, reliable, and valid; and (3) in comparison to motion capture devices, no high-resolution camera equipment is necessary, and no sensors are attached to the patient's body (Altmann, 2010; Ramseyer & Tschacher, 2011). Therefore, during the past few years, the use of MEA has become enormously widespread. In behavioral and social science, MEA has been used to assess movements in mother-child interactions (

Reduced nonverbal interpersonal synchrony in autism spectrum disorder independent of partner diagnosis: a motion energy study

“…Moreover, our approach is only one of many existing methods used to measure IPS. Schoenherr and colleagues [18] have recently reviewed several linear methods of time series analysis that can be used to quantify IPS and find that they address different aspects of coordination. In addition, nonlinear methods have been used to quantify how and the extent to which streams of information come to exhibit similar patterns in time [15,[114][115][116].…”

“…In this article, we will use the term interpersonal synchrony (IPS) to denote this type of coordination. It can include a broad range of nonverbal behaviours and ranges from simultaneous occurrence of behaviours to behaviours involving a short delay [17,18]. IPS may underlie interaction success because it can promote effective turn-taking, as well as connectedness, trust and prosocial behaviour [11,19], and it can even predict the success of a problem-solving exercise, negotiation or meeting [20,21].…”

Quantifying Physiological Synchrony through Windowed Cross-Correlation Analysis: Statistical and Theoretical Considerations

“…While one method might be appropriate for one, it might not be for another depending on, among others, the type of data (e.g., continuous or categorical measures) and the measure of interest (e.g., strengths versus frequency of synchrony; global versus time-sensitive measure of synchrony) (Gates & Liu, 2016;Schoenherr et al, 2018). For example, we chose to treat facial muscle activity as a continuous measure.…”

“…A variety of methods have been proposed in previous literature to quantify synchrony including correlations, regressions, structural equation models and recurrence quantification analyses. These approaches differ in their assumptions, their operationalization of synchrony, and the type of synchrony they measure (for a review, see Gates & Liu, 2016;McAssey, Helm, Hsieh, Sbarra, & Ferrer, 2013;Schoenherr et al, 2018;Thorson, West, & Mendes, 2017). In the current article, we focus on continuous time series measures in dyads.…”

“…The WCC analysis requires the specification of four parameters that tailors the method to the signal of interest: window size, maximum lag, window increment, and lag increment (see Figure 1). Carefully choosing the right parameter settings is crucial, because these settings can substantially affect the outcome of the WCC analysis (Schoenherr et al, 2018).…”