Enhanced neural tracking of the fundamental frequency of the voice

Canneyt, Jana Van; Wouters, Jan; Francart, Tom

doi:10.1101/2020.10.28.359034

Cited by 4 publications

(3 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…size f = 0.98; imagery: p = 2.7 Â 10 -209 common language effect size f = 0.97; see Materials and Methods). The note onset encoding was significant at the individual participant level (17 of 21, p , 0.05, FDR-corrected p values extracted from the null model distributions) and was most accurately encoded on central scalp areas, as previously shown in response to auditory experiments (Di Liberto et al, 2020a,b;Van Canneyt et al, 2020). A significant (p = 0.02) correlation of r = 0.3 was measured between the topographies of the EEG prediction values for the two conditions (Pearson's correlation).…”

Section: Onsets Encodingsupporting

confidence: 71%

The Music of Silence. Part I: Responses to Musical Imagery Encode Melodic Expectations and Acoustics

2021

View full text Add to dashboard Cite

Musical imagery is the voluntary internal hearing of music in the mind without the need for physical action or external stimulation. Numerous studies have already revealed brain areas activated during imagery. However, it remains unclear to what extent imagined music responses preserve the detailed temporal dynamics of the acoustic stimulus envelope and, crucially, whether melodic expectations play any role in modulating responses to imagined music, as they prominently do during listening. These modulations are important as they reflect aspects of the human musical experience, such as its acquisition, engagement, and enjoyment. This study explored the nature of these modulations in imagined music based on EEG recordings from 21 professional musicians (6 females and 15 males). Regression analyses were conducted to demonstrate that imagined neural signals can be predicted accurately, similarly to the listening task, and were sufficiently robust to allow for accurate identification of the imagined musical piece from the EEG. In doing so, our results indicate that imagery and listening tasks elicited an overlapping but distinctive topography of neural responses to sound acoustics, which is in line with previous fMRI literature. Melodic expectation, however, evoked very similar frontal spatial activation in both conditions, suggesting that they are supported by the same underlying mechanisms. Finally, neural responses induced by imagery exhibited a specific transformation from the listening condition, which primarily included a relative delay and a polarity inversion of the response. This transformation demonstrates the top-down predictive nature of the expectation mechanisms arising during both listening and imagery.

show abstract

Section: Onsets Encodingsupporting

confidence: 71%

The Music of Silence. Part I: Responses to Musical Imagery Encode Melodic Expectations and Acoustics

2021

View full text Add to dashboard Cite

show abstract

“…More complicated and computationally expensive techniques have also been explored, including empirical mode decomposition (Etard et al, 2019; Forte et al, 2017) and auditory modelling (Van Canneyt et al, 2021b). Constructing an f0 feature that approximates the expected neural response using auditory modelling has proven particularly effective, nearly doubling the reconstruction accuracies obtained with the neural tracking analysis (Van Canneyt et al, 2021b). This is likely explained by the fact that the auditory model is more physiologically valid.…”

Section: Neural Tracking Of the F0mentioning

confidence: 99%

Neural tracking as a diagnostic tool to assess the auditory pathway

Canneyt

Gillis

Vanthornhout

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The neural tracking framework enables the analysis of neural responses (EEG) to continuous natural speech, e.g., a story or a podcast. This allows for objective investigation of a range of auditory and linguistic processes in the brain during natural speech perception. This approach is more ecologically valid than traditional auditory evoked responses and has great potential for both research and clinical applications. In this article, we review the neural tracking framework and highlight three prominent examples of neural tracking analyses. This includes the neural tracking of the fundamental frequency of the voice (f0), the speech envelope and linguistic features. Each of these analyses provides a unique point of view into the hierarchical stages of speech processing in the human brain. f0-tracking assesses the encoding of fine temporal information in the early stages of the auditory pathway, i.e. from the auditory periphery up to early processing in the primary auditory cortex. This fundamental processing in (mostly) subcortical stages forms the foundation of speech perception in the cortex. Envelope tracking reflects bottom-up and top-down speech-related processes in the auditory cortex, and is likely necessary but not sufficient for speech intelligibility. To study neural processes more directly related to speech intelligibility, neural tracking of linguistic features can be used. This analysis focuses on the encoding of linguistic features (e.g. word or phoneme surprisal) in the brain. Together these analyses form a multi-faceted and time-effective objective assessment of the auditory and linguistic processing of an individual.

show abstract

“…The original algorithm for computing the fundamental waveform was based on empirical mode decomposition (Huang and Pan (2006); Forte et al (2017)). However, Etard et al (2019) showed that direct band-pass filtering of the speech signal is considerably simpler, faster to compute and leads to the same result (Kulasingham et al (2020); Van Canneyt et al (2021b,a); Bachmann et al (2021)). Here, we also employed a band-pass filter to extract the fundamental waveform from the voice recordings.…”

Section: Methodsmentioning

confidence: 99%

The Neural Response at the Fundamental Frequency of Speech is Modulated by Word-level Acoustic and Linguistic Information

Kegler

Weissbart

Reichenbach

2022

Preprint

View full text Add to dashboard Cite

Spoken language comprehension requires rapid and continuous integration of information, from lower-level acoustic to higher-level linguistic features. Much of this processing occurs in the cerebral cortex. Its neural activity exhibits, for instance, correlates of predictive processing, emerging at delays of a few hundred milliseconds. However, the auditory pathways are also characterized by extensive feedback loops from higher-level cortical areas to lower-level ones as well as to subcortical structures. Early neural activity can therefore be influenced by higher-level cognitive processes, but it remains unclear whether such feedback contributes to linguistic processing. Here, we investigated early speech-evoked neural activity that emerges at the fundamental frequency. We analyzed EEG recordings obtained when subjects listened to a story read by a single speaker. We identified a response tracking the speaker's fundamental frequency that occurred at a delay of 11 ms, while another response elicited by the high-frequency modulation of the envelope of higher harmonics exhibited a larger magnitude and longer latency of about 18 ms. Subsequently, we determined the magnitude of these early neural responses for each individual word in the story. We then quantified the context-independent frequency of each word and used a language model to compute context-dependent word surprisal and precision. The word surprisal represented how predictable a word is, given the previous context, and the word precision reflected the confidence about predicting the next word from the past context. We found that the word-level neural responses at the fundamental frequency were predominantly influenced by the acoustic features: the average fundamental frequency and its variability. Amongst the linguistic features, only context-independent word frequency showed a weak but significant modulation of the neural response to the high-frequency envelope modulation. Our results show that the early neural response at the fundamental frequency is already influenced by acoustic as well as linguistic information, suggesting top-down modulation of this neural response.

show abstract

Enhanced neural tracking of the fundamental frequency of the voice

Cited by 4 publications

References 41 publications

The Music of Silence. Part I: Responses to Musical Imagery Encode Melodic Expectations and Acoustics

The Music of Silence. Part I: Responses to Musical Imagery Encode Melodic Expectations and Acoustics

Neural tracking as a diagnostic tool to assess the auditory pathway

The Neural Response at the Fundamental Frequency of Speech is Modulated by Word-level Acoustic and Linguistic Information

Contact Info

Product

Resources

About