Finding Structure in Time: Visualizing and Analyzing Behavioral Time Series

Abstract

The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the non-linear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique's application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a "Programming Basics" module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data.

Using Vector Autoregression Modeling to Reveal Bidirectional Relationships in Gender/Sex-Related Interactions in Mother–Infant Dyads

“…VAR estimates connection strengths between each pair of nodes, i.e., each pair of behaviors-providing ultimately a web (Figure 1) of relationships summarizing a complex system. It is important to guard always against misconstruing the relationships in a VAR as explicitly causal, but they offer the potential for testing whether two variables are related in an extremely limited but still interesting type of cause, namely, Granger cause, in which the prior changes of one variable are associated with later changes in a second variable above and beyond the second variable's contributions to its own later changes (see Roebroeck et al, 2005;Xu et al, 2020).…”

“…We have in mind for the present manuscript a method called "vector autoregression" (VAR; Sims, 1980) that, despite having enjoyed rich elaboration and wide application over the past few decades (Lutkepohl, 2005), has remained out of the view of some developmental psychologists. This incomplete appearance of VAR to psychologists is likely due to the fact that these methods arose in econometrics and have only appeared in a few subfields of psychology also interested in bidirectional relationships (Roebroeck et al, 2005;Billinger et al, 2014;van Winkel et al, 2017;Xu et al, 2020). In this paper, we will showcase VAR for a developmental-psychological audience.…”

Interpersonal synchrony is associated with infants’ reactions to subtle changes in caregiver-infant interactions

“…CRQA is able to detect behavioural attunement and coordination in non-linear nominal time series and has been increasingly employed in analyses of behavioural patterns (Abney et al, 2015;Nomikou et al, 2016). The behavioural time-series were downsampled from 25 Hz to 5 Hz and the recurrence rate (i.e., the proportion of behavioural matches between gazes at the respective partner or mutual positive affect) was estimated in Matlab using the Chromatic_CRQA function (Xu et al, 2020).…”

“Dancing” Together: Infant–Mother Locomotor Synchrony

“…For example, if the mother's future location is significantly better predicted by her location history and her infant's location history than her location history alone, this means that the infant's location history contains additional information that helps to predict the mother's future location above and beyond her own location history. In this case, then the infant's location is said to Granger-cause the mother's location (for a tutorial using continuous values see Barnett & Seth, 2014; for a tutorial using discrete binary spike trains or point processes see Xu, de Barbaro, Abney, & Cox, 2019).…”