An Analytical Framework of Tonal and Rhythmic Hierarchy in Natural Music Using the Multivariate Temporal Response Function

Leahy, J Clancy; Kim, Seung‐Goo; Wan, Jie; Overath, Tobias

doi:10.3389/fnins.2021.665767

Cited by 9 publications

(8 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The inverse-correlation approach used to estimate the TRFs is highly affected by the degree of autocorrelation in the stimulus, and hence the two speech stimuli are not directly comparable (Crosse et al, 2016). Nonetheless, even highly rhythmic stimuli have been shown to produce reliable TRFs in other studies (Leahy et al, 2021;Lerousseau et al, 2021), therefore, this methodological caveat alone probably does not account for the lack of any reliable signature of neural tracking of the Barista Stream observed here.…”

Section: Processing Concurrent Speech -Neural Prioritization Without ...mentioning

confidence: 79%

An ecological investigation of the capacity to follow simultaneous speech and preferential detection of ones’ own name

Pinto

Kaufman

Brown

et al. 2022

Preprint

View full text Add to dashboard Cite

Many situations require focusing attention on one speaker, while monitoring the environment for potentially important information. Some have proposed that dividing attention among two speakers involves behavioral tradeoffs, due to limited cognitive resources. However the severity of these tradeoffs, particularly under ecologically-valid circumstances, is not well understood. We investigated the capacity to process simultaneous speech using a dual-task paradigm simulating task demands and stimuli encountered in real-life. Participants listened to conversational narratives (Narrative Stream) and monitored a stream of announcements (Barista Stream), to detect when their order was called. We measured participants' performance, neural activity and skin conductance as they engaged in this dual-task. Participants achieved extremely high dual-task accuracy, with no apparent behavioral tradeoffs. Moreover, robust neural and physiological responses were observed for target-stimuli in the Barista Stream, alongside significant neural speech-tracking of the Narrative Stream. These results suggest that humans have substantial capacity to process simultaneous speech and do not suffer from insufficient processing resources, at least for this highly ecological task-combination and level of perceptual load. Results also confirmed the ecological validity of the advantage for detecting ones' own name at the behavioral, neural and physiological level, highlighting the contribution of personal relevance when processing simultaneous speech.

show abstract

Section: Processing Concurrent Speech -Neural Prioritization Without ...mentioning

confidence: 79%

An ecological investigation of the capacity to follow simultaneous speech and preferential detection of ones’ own name

Pinto

Kaufman

Brown

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While cortical processing of speech has been a primary application for mTRF analysis, the technique has potential applications in any domain where stimuli unfold in time, and has already been successfully applied to music perception (Di Liberto et al, 2020; Leahy et al, 2021), audiovisual speech perception (Crosse et al, 2016a), and subcortical auditory processing (Maddox and Lee, 2018). Furthermore, mTRF analysis as discussed here assumes that TRFs are static across time.…”

Section: Discussionmentioning

confidence: 99%

Eelbrain: A Python toolkit for time-continuous analysis with temporal response functions

Brodbeck¹,

Das

Kulasingham

et al. 2021

Preprint

View full text Add to dashboard Cite

Even though human experience unfolds continuously in time, it is not strictly linear; instead, it entails cascading processes building hierarchical cognitive structures. For instance, during speech perception, humans transform a continuously varying acoustic signal into phonemes, words, and meaning, and these levels all have different, interdependent temporal structures. Deconvolution analysis has recently emerged as a promising tool for disentangling electrophysiological brain responses related to such complex models of perception. Here we introduce the Eelbrain Python toolkit, which makes this kind of analysis easy and accessible. We demonstrate its use, using continuous speech as a sample paradigm, with a freely available EEG dataset of audiobook listening. A companion GitHub repository provides the complete source code for the analysis, from raw data to group level statistics. More generally, we advocate a hypothesis-driven approach in which the experimenter specifies a hierarchy of time-continuous representations that are hypothesized to have contributed to brain responses, and uses those as predictor variables for the electrophysiological signal. This is analogous to a multiple regression problem, but with the addition of the time dimension. The deconvolution analysis decomposes the brain signal into distinct responses associated with the different predictor variables by estimating a multivariate temporal response function (mTRF), quantifying the influence of each predictor on brain responses as a function of time(-lags). This allows asking two questions about the predictor variables: 1) Is there a significant neural representation corresponding to this predictor variable? And if so, 2) what are the temporal characteristics of the neural response associated with it? Thus, different predictor variables can be systematically combined and evaluated to jointly model neural processing at multiple hierarchical levels. We discuss applications of this approach, including the potential for linking algorithmic/representational theories at different cognitive levels to brain responses through appropriate linking models.

show abstract

“…One possible avenue of future research might include using regression or CCA to decode the relationship between pupillary activity and the amplitude envelope of speech or music (N.B. one could do this for any number of different stimulus features of interest, see e.g., Leahy et al, 2021 ). Further, by estimating a TRF for each participant individually, one could then compare differences in the parameters of TRFs between participants to determine the degree of pupil response variability for some particular stimulus feature.…”

Section: Analysis Techniquesmentioning

confidence: 99%

From pre-processing to advanced dynamic modeling of pupil data

et al. 2023

View full text Add to dashboard Cite

The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants’ pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.

show abstract

An Analytical Framework of Tonal and Rhythmic Hierarchy in Natural Music Using the Multivariate Temporal Response Function

Cited by 9 publications

References 60 publications

An ecological investigation of the capacity to follow simultaneous speech and preferential detection of ones’ own name

An ecological investigation of the capacity to follow simultaneous speech and preferential detection of ones’ own name

Eelbrain: A Python toolkit for time-continuous analysis with temporal response functions

From pre-processing to advanced dynamic modeling of pupil data

Contact Info

Product

Resources

About