Comparing the error‐related negativity across groups: The impact of error‐ and trial‐number differences

Abstract

The error-related negativity (ERN or Ne) is increasingly being investigated as a marker discriminating interindividual factors and moves toward a surrogate marker for disorders or interventions. Although reproducibility and validity of neuroscientific and psychological research has been criticized, clear data on how different quantification methods of the ERN and their relation to available trial numbers affect within- and across-participant studies is sparse. Within a large sample of 863 healthy human participants, we demonstrate that, across participants, the number of errors correlates with the amplitude of the ERN independently of the number of errors included in ERN quantification per participant, constituting a possible confound when such variance is unaccounted for. Additionally, we find that ERN amplitudes reach high consistency within participants at lower trial numbers, yet when comparisons between groups of participants are desired, increasing error-trial numbers lead to higher statistical power. We derive concrete suggestions for specific types of analyses, which may help researchers to more effectively design studies and analyze error-related EEG data with the most appropriate measurement technique for the question at hand and trial number available.

How many trials does it take to get a significant ERP effect? It depends

“…That is, for the ERN we examined the likelihood of finding a statistically significant amplitude difference between error trials and correct trials; for the LRP, we examined the likelihood of finding a statistically significant amplitude difference between the hemispheres contralateral versus ipsilateral to the response hand. Consistent with previous work (Fischer et al, 2017;Larson et al, 2010;Olvet & Hajcak, 2009;Pontifex et al, 2010;Steele et al, 2016), the ERN proved to be a robust effect that was reliably observed even when relatively few trials were included in the average (see Figure 6a). In addition, the internal reliability of the ERN reached the "high" range (0.7-0.9) with only 8 error trials, which also replicates prior results (Olvet & Hajcak, 2009).…”

“…However, as noted by Gehring, Liu, Orr, and Carp (2012, p. 278), "this does not speak to the ability of standard analyses to find between-condition or betweengroup differences." This point has also been made by a recent study, in which the ability to detect a between-groups difference on the ERN was examined as a function of the number of trials included in the ERP average and the error rate across groups (Duncan et al, 2009;Fischer et al, 2017). The study by Fischer et al offered some initial evidence that specific recommendations that are based on tests of ERP stability may not be appropriate as guidelines for detecting differences between groups.…”

“…More recently, studies have attempted to provide specific, data-driven guidelines for how many trials should be included in an ERP experiment. These studies have examined several widely used ERP components, including the error-related negativity (ERN), error positivity (Pe), N100, N200, vertex positive potential (VPP)/N170, mismatch-negativity (MMN), feedback-related negativity (FRN), late positive potential (LPP), and P300 (Cohen & Polich, 1997; Duncan et al, 2009; Fischer, Klein, & Ullsperger, 2017; Huffmeijer, Bakermans-Kranenburg, Alink, & van IJzendoorn, 2014; Larson, Baldwin, Good, & Fair, 2010; Marco-Pallares, Cucurell, Münte, Strien, & Rodriguez-Fornells, 2011; Olvet & Hajcak, 2009; Pontifex et al, 2010; Rietdijk, Franken, & Thurik, 2014; Segalowitz & Barnes, 1993; Steele et al, 2016; Thigpen, Kappenman, & Keil, 2017). The general approach of these studies has been to take the data from an experiment with a particular number of trials and simulate experiments with smaller numbers of trials by subsampling from the original dataset.…”

“…An important conclusion that one might draw from previous studies of the number of trials is that ERP researchers can collect data from relatively small numbers of trials without any practical cost to data quality (Cohen & Polich, 1997; Duncan et al, 2009; Fischer et al, 2017; Huffmeijer et al, 2014; Larson et al, 2010; Marco-Pallares et al, 2011; Olvet & Hajcak, 2009; Pontifex et al, 2010; Rietdijk et al, 2014; Segalowitz & Barnes, 1993; Steele et al, 2016; Thigpen et al, 2017). For example, a researcher may read that “P300 amplitude stabilizes with approximately 20 target trials for all conditions” (Cohen & Polich, 1997, p. 249) or that the “ERN and Pe may be accurately quantified with as few as six to eight commission error trials across the life span” (Pontifex et al, 2010, p. 767).…”

Null results from a dimensional study of error-related negativity (ERN) and self-reported psychiatric symptoms

“…Psychiatric symptoms and transdiagnostic dimensions. We tested if ERN amplitudes were associated to the self-reported symptom scores including error rate (β = 0.40, SE = 0.20, p < 0.04) as a control co-variate, which previously have been shown to influence ERN amplitudes (Fischer et al, 2017). In contrast to our hypothesis, none of the psychiatric questionnaires showed a significant relationship to ERN amplitude (all p > 0.13, where p < 0.005 is the Bonferroni corrected significance threshold) ( Figure 2 and Table 1).…”