Looking for consistency in an uncertain world: test-retest reliability of neurophysiological and behavioral readouts in autism

Beker, Shlomit; Foxe, John J.; Venticinque, John; Bates, Juliana C.; Ridgeway, Elizabeth M.; Schaaf, Roseann C.; Molholm, Sophie

doi:10.1186/s11689-021-09383-0

Cited by 8 publications

(5 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is achieved by assaying the latencies and amplitudes of well-characterized event-related potential (ERP) components, which have stereotypical topology and temporal dynamics in neurotypical populations, and have been well characterized in thousands of papers over the past 60 years (Chope, Metz-Lutz et al 1994) (Luck 2014) (Simpson, Pflieger et al 1995) (Martin, Barajas et al 1988) (Sutton, Tueting et al 1967) (Sutton, Braren et al 1965) (Ritter and Vaughan 1969). A high degree of test-retest reliability is also a feature of this method, making it ideal for longitudinal monitoring of intervention trials (Kileny and Kripal 1987) (Malcolm, Foxe et al 2019) (Beker, Foxe et al 2021).…”

Section: Introductionmentioning

confidence: 99%

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome

Brima,

Beker,

Prinsloo

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Background: In the search for objective tools to quantify neural function in Rett Syndrome (RTT), which are crucial in the evaluation of therapeutic efficacy in clinical trials, recordings of sensory-perceptual functioning using event-related potential (ERP) approaches have emerged as potentially powerful tools. Considerable work points to highly anomalous auditory evoked potentials (AEPs) in RTT. However, an assumption of the typical signal-averaging method used to derive these measures is "stationarity" of the underlying responses - i.e. neural responses to each input are highly stereotyped. An alternate possibility is that responses to repeated stimuli are highly variable in RTT. If so, this will significantly impact the validity of assumptions about underlying neural dysfunction, and likely lead to overestimation of underlying neuropathology. To assess this possibility, analyses at the single-trial level assessing signal-to-noise ratios (SNR), inter-trial variability (ITV) and inter-trial phase coherence (ITPC) are necessary. Methods: AEPs were recorded to simple 100Hz tones from 18 RTT and 27 age-matched controls (Ages: 6-22 years). We applied standard AEP averaging, as well as measures of neuronal reliability at the single-trial level (i.e. SNR, ITV, ITPC). To separate signal-carrying components from non-neural noise sources, we also applied a denoising source separation (DSS) algorithm and then repeated the reliability measures. Results: Substantially increased ITV, lower SNRs, and reduced ITPC were observed in auditory responses of RTT participants, supporting a "neural unreliability" account. Application of the DSS technique made it clear that non-neural noise sources contribute to overestimation of the extent of processing deficits in RTT. Post-DSS, ITV measures were substantially reduced, so much so that pre-DSS ITV differences between RTT and TD populations were no longer detected. In the case of SNR and ITPC, DSS substantially improved these estimates in the RTT population, but robust differences between RTT and TD were still fully evident. Conclusions: To accurately represent the degree of neural dysfunction in RTT using the ERP technique, a consideration of response reliability at the single-trial level is highly advised. Non-neural sources of noise lead to overestimation of the degree of pathological processing in RTT, and denoising source separation techniques during signal processing substantially ameliorate this issue.

show abstract

Section: Introductionmentioning

confidence: 99%

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome

Brima,

Beker,

Prinsloo

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is achieved by assaying the latencies and amplitudes of well-characterized event-related potential (ERP) components, which have stereotypical topology and temporal dynamics in neurotypical populations, and have been well characterized in thousands of papers over the past 60 years [ 14 , 39 , 44 , 54 , 62 , 65 , 66 ]. A high degree of test–retest reliability is also a feature of this method, making it ideal for longitudinal monitoring of intervention trials [ 5 , 34 , 42 ]. However, a central assumption of this methodology is stationarity of response – that is, that when a stimulus is presented repeatedly to a participant, the neural response on each iteration (or trial) is assumed to be essentially identical, whereby the simple process of signal-averaging across trials will reveal this stationary canonical response because temporally random background activity (noise) will be eliminated through the averaging procedure [ 30 , 39 , 54 ].…”

Section: Introductionmentioning

confidence: 99%

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome

Brima,

Beker,

Prinsloo

et al. 2024

J Neurodevelop Disord

Self Cite

View full text Add to dashboard Cite

Background In the search for objective tools to quantify neural function in Rett Syndrome (RTT), which are crucial in the evaluation of therapeutic efficacy in clinical trials, recordings of sensory-perceptual functioning using event-related potential (ERP) approaches have emerged as potentially powerful tools. Considerable work points to highly anomalous auditory evoked potentials (AEPs) in RTT. However, an assumption of the typical signal-averaging method used to derive these measures is “stationarity” of the underlying responses – i.e. neural responses to each input are highly stereotyped. An alternate possibility is that responses to repeated stimuli are highly variable in RTT. If so, this will significantly impact the validity of assumptions about underlying neural dysfunction, and likely lead to overestimation of underlying neuropathology. To assess this possibility, analyses at the single-trial level assessing signal-to-noise ratios (SNR), inter-trial variability (ITV) and inter-trial phase coherence (ITPC) are necessary. Methods AEPs were recorded to simple 100 Hz tones from 18 RTT and 27 age-matched controls (Ages: 6–22 years). We applied standard AEP averaging, as well as measures of neuronal reliability at the single-trial level (i.e. SNR, ITV, ITPC). To separate signal-carrying components from non-neural noise sources, we also applied a denoising source separation (DSS) algorithm and then repeated the reliability measures. Results Substantially increased ITV, lower SNRs, and reduced ITPC were observed in auditory responses of RTT participants, supporting a “neural unreliability” account. Application of the DSS technique made it clear that non-neural noise sources contribute to overestimation of the extent of processing deficits in RTT. Post-DSS, ITV measures were substantially reduced, so much so that pre-DSS ITV differences between RTT and TD populations were no longer detected. In the case of SNR and ITPC, DSS substantially improved these estimates in the RTT population, but robust differences between RTT and TD were still fully evident. Conclusions To accurately represent the degree of neural dysfunction in RTT using the ERP technique, a consideration of response reliability at the single-trial level is highly advised. Non-neural sources of noise lead to overestimation of the degree of pathological processing in RTT, and denoising source separation techniques during signal processing substantially ameliorate this issue.

show abstract

“…This is achieved by assaying the latencies and amplitudes of wellcharacterized event-related potential (ERP) components, which have stereotypical topology and temporal dynamics in neurotypical populations, and have been well characterized in thousands of papers over the past 60 years (Chope, Metz-Lutz et al 1994) (Luck 2014) (Simpson, P ieger et al 1995) (Martin, Barajas et al 1988) (Sutton, Tueting et al 1967) (Sutton, Braren et al 1965) (Ritter and Vaughan 1969). A high degree of test-retest reliability is also a feature of this method, making it ideal for longitudinal monitoring of intervention trials (Kileny and Kripal 1987) (Malcolm, Foxe et al 2019) (Beker, Foxe et al 2021).…”

Section: Introductionmentioning

confidence: 99%

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome.

Brima,

Beker,

Prinsloo

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Background In the search for objective tools to quantify neural function in Rett Syndrome (RTT), which are crucial in the evaluation of therapeutic efficacy in clinical trials, recordings of sensory-perceptual functioning using event-related potential (ERP) approaches have emerged as potentially powerful tools. Considerable work points to highly anomalous auditory evoked potentials (AEPs) in RTT. However, an assumption of the typical signal-averaging method used to derive these measures is “stationarity” of the underlying responses – i.e. neural responses to each input are highly stereotyped. An alternate possibility is that responses to repeated stimuli are highly variable in RTT. If so, this will significantly impact the validity of assumptions about underlying neural dysfunction, and likely lead to overestimation of underlying neuropathology. To assess this possibility, analyses at the single-trial level assessing signal-to-noise ratios (SNR), inter-trial variability (ITV) and inter-trial phase coherence (ITPC) are necessary. Methods AEPs were recorded to simple 100Hz tones from 18 RTT and 27 age-matched controls (Ages: 6–22 years). We applied standard AEP averaging, as well as measures of neuronal reliability at the single-trial level (i.e. SNR, ITV, ITPC). To separate signal-carrying components from non-neural noise sources, we also applied a denoising source separation (DSS) algorithm and then repeated the reliability measures. Results Substantially increased ITV, lower SNRs, and reduced ITPC were observed in auditory responses of RTT participants, supporting a “neural unreliability” account. Application of the DSS technique made it clear that non-neural noise sources contribute to overestimation of the extent of processing deficits in RTT. Post-DSS, ITV measures were substantially reduced, so much so that pre-DSS ITV differences between RTT and TD populations were no longer detected. In the case of SNR and ITPC, DSS substantially improved these estimates in the RTT population, but robust differences between RTT and TD were still fully evident. Conclusions To accurately represent the degree of neural dysfunction in RTT using the ERP technique, a consideration of response reliability at the single-trial level is highly advised. Non-neural sources of noise lead to overestimation of the degree of pathological processing in RTT, and denoising source separation techniques during signal processing substantially ameliorate this issue.

show abstract

Looking for consistency in an uncertain world: test-retest reliability of neurophysiological and behavioral readouts in autism

Cited by 8 publications

References 82 publications

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome

Probing a neural unreliability account of auditory sensory processing atypicalities in Rett Syndrome.

Contact Info

Product

Resources

About