Bootstrap Signal-to-Noise Confidence Intervals: An Objective Method for Subject Exclusion and Quality Control in ERP Studies

Abstract

Analysis of event-related potential (ERP) data includes several steps to ensure that ERPs meet an appropriate level of signal quality. One such step, subject exclusion, rejects subject data if ERP waveforms fail to meet an appropriate level of signal quality. Subject exclusion is an important quality control step in the ERP analysis pipeline as it ensures that statistical inference is based only upon those subjects exhibiting clear evoked brain responses. This critical quality control step is most often performed simply through visual inspection of subject-level ERPs by investigators. Such an approach is qualitative, subjective, and susceptible to investigator bias, as there are no standards as to what constitutes an ERP of sufficient signal quality. Here, we describe a standardized and objective method for quantifying waveform quality in individual subjects and establishing criteria for subject exclusion. The approach uses bootstrap resampling of ERP waveforms (from a pool of all available trials) to compute a signal-to-noise ratio confidence interval (SNR-CI) for individual subject waveforms. The lower bound of this SNR-CI (SNRLB) yields an effective and objective measure of signal quality as it ensures that ERP waveforms statistically exceed a desired signal-to-noise criterion. SNRLB provides a quantifiable metric of individual subject ERP quality and eliminates the need for subjective evaluation of waveform quality by the investigator. We detail the SNR-CI methodology, establish the efficacy of employing this approach with Monte Carlo simulations, and demonstrate its utility in practice when applied to ERP datasets.

Excitability regulation in the dorsomedial prefrontal cortex during sustained instructed fear responses: a TMS-EEG study

“…To ascertain the appropriate signal quality of the evoked potentials the SNR from a pool of all available trials for individual subject waveforms was computed as: where TL signal is the mean activity within a time window of 500 ms containing the evoked waves, TL noise is the average signal at baseline, and SNR EP is the SNR for the time window of interest expressed in decibels (dB), providing a straightforward measure that quantifies the signal strength of an evoked waveform 61 . A time window was used instead of the amplitude peaks since a single point estimate of SNR cannot fully portray the quality of an evoked activity, as it does not capture the variability of the signals 61 . Therefore, this calculation of SNR on signal averages is the most accurate representation of the evoked quality.…”

Depression-Related Brain Connectivity Analyzed by EEG Event-Related Phase Synchrony Measure

“…To identify the task-dependent modulation of the PSI, we compared the PSIs of the individual participants between the target and nontarget conditions with a two-sample t -test. To estimate the confidence interval for the t -value, we used a bootstrap procedure (Mizuhara et al, 2005 ; Parks et al, 2016 ), which frees us from making unverifiable assumptions about the data (e.g., probability distribution). Using the 2000 bootstrapped re-samples that were obtained from PSI a,b ( f,t ) in the target and nontarget conditions, the t -values were computed with the two-sampled t -test for each individual participant.…”

Printable microscale interfaces for long-term peripheral nerve mapping and precision control

“…. We considered an evoked response to be detected if the SNR within the signal response window exceeded a 90% confidence interval calculated by bootstrap (i.e., resampling with replacement the signal and noise intervals over n = 10000 trials) (Parks et al, 2016). Figures 3b and 5b show individual stimulation trials from single experimental sessions.…”