Discovering precise temporal patterns in large-scale neural recordings through robust and interpretable time warping

Abstract

1Though the temporal precision of neural computation has been studied intensively, a data-driven determination 2 of this precision remains a fundamental challenge. Reproducible spike time patterns may be obscured on single 3 trials by uncontrolled temporal variability in behavior and cognition, or may not even be time locked to measurable 4 signatures in either behavior or local field potentials (LFP). To overcome these challenges, we describe a general-5 purpose time warping framework that reveals precise spike-time patterns in an unsupervised manner, even when 6 spiking is decoupled from behavior or is temporally stretched across single trials. We demonstrate this method 7 across diverse systems: cued reaching in nonhuman primates, motor sequence production in rats, and olfaction in 8 mice. This approach flexibly uncovers diverse dynamical firing patterns, including pulsatile responses to behavioral 9 events, LFP-aligned oscillatory spiking, and even unanticipated patterns, like 7 Hz oscillations in rat motor cortex 10 that are not time-locked to measured behaviors or LFP. 11 14 Amarasingham et al. 2015; Brette 2015; Denève and Machens 2016), engendering intense debates in systems 15neuroscience over the last several decades. Empirically determining the degree of temporal precision from data is 16 challenging because multi-neuronal spike trains may contain highly structured temporal patterns that are completely 17 1 masked by temporal variations in behavioral and cognitive variables not under direct experimental control. For 18 example, precise spike patterns may not be temporally locked to naïvely chosen sensory or behavioral events. 19 Indeed, the fidelity of olfactory coding may be underestimated by factors of two to four when spike times are aligned 20 to stimulus delivery instead of inhalation onset (Shusterman et al. 2011; Cury and Uchida 2010; Shusterman et al. 21 2018). 22 Thus, experimental estimates of spike time precision hinge on the choice of an alignment point, which defines the 23 origin of the time axis on each trial. This choice can often be challenging and subjective. Even in relatively simple 24 behavioral tasks, animals can experience a sequence of stimuli, actions, and rewards, each of which occur with 25 varying latencies on different trials. Such tasks thus provide multiple choices for aligning multineuronal spike trains 26 to measurable events marking an origin of time. Moreover, in addition to choosing an origin of time, we must also 27 choose its units. Should spike times be measured in absolute clock time relative to some measured event, or in 28 units of fractional time between two events? Should the units of time change between successive pairs of events? 29 Could any one of these choices unmask spike-timing precision that is otherwise invisible? 30 Past studies have addressed these challenges in a number of ways: grouping trials together with similar durations 31 before averaging spike counts (Murakami et al. 2014; Starkweather et al. 2017; Wang et al. 2018), manually 32...

High-performance brain-to-text communication via imagined handwriting

“…Principal components analysis was then applied to this matrix to find the top 3 PCs which were used to visualize the raw activity ( Figure 1B). Next, we used time-warped PCA (https://github.com/ganguli-lab/twpca) 11,33 to find continuous, regularized time-warping functions that align the trials within a single movement condition together. We verified that these warping functions appeared close to the identity line, smoothly bending away from it after the go cue in order to account for variations in writing speed from trial to trial (as can be seen in the example shown in Figure 1B-C).…”

“…The neural activity appeared to be strong and repeatable, although the timing of its peaks and valleys varied across trials (potentially due to fluctuations in writing speed). We used a timealignment technique to remove temporal variability 11 , revealing remarkably consistent underlying patterns of neural activity that are unique to each character (Fig. 1C).…”

“…(A) We analyzed the spatiotemporal patterns of neural activity corresponding to handwritten characters (1 second in duration) vs. 16handwritten straight-line movements (0.6 seconds in duration). (B) Spatiotemporal neural patterns were found by averaging over all trials for a given movement (after time-warping to align the trials in time)11 . Neural activity was then resampled to equalize the duration of each set of movements (otherwise straight-line movements would beshorter in duration), resulting in a 192 x 100 matrix for each movement (192 electrodes and 100 time steps).…”

Synergistic population encoding and precise coordinated variability across interlaminar ensembles in the early visual system

“…To directly address the question of correlated jitter within early visual populations, we used a measure inspired by spike count noise correlation: the Spearman correlation (ρ) of jitter across trials (henceforth notated ρjitter). To quantify jitter for each cell, we use a Dynamic Time Warping (DTW) procedure adapted from 61 ( Figure 4A and see Methods). Briefly, a linear time warping procedure is used to fit a temporal shift for each trial that best aligns that trial with all of the other trials via a least-squares loss.…”

“…Conversely, if one cell tends to "early" when the other is "late", they have a negative ρjitter. The linear DTW method straightforwardly extends to populations to estimate population shifts (ref 61 , Fig 4A right side). We analyzed the correlation of the trial-to-trial shifts estimated from single cells with other single cells (ρjitter) and single cells and variously constructed sub-populations of simultaneously recorded cells (ρglobal jitter).…”