Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting

Clayson, Peter E.; Miller, Gregory A.

doi:10.1016/j.ijpsycho.2016.09.005

Cited by 117 publications

(131 citation statements)

References 93 publications

Supporting

Mentioning

128

Contrasting

Order By: Relevance

“…This study used the doors task to examine the internal consistency and test‐retest reliability of the RewP and FN to monetary gains and losses in a sample of young adults. Consistent with previously reported findings (Bress et al, ), both the RewP and FN achieved good to excellent internal reliability (all values > .7) within 20 trials of the doors task at both assessments based on split‐half reliability, Cronbach's α, and G theory measures of overall dependability (Clayson & Miller, ; Gliem & Gliem, ). The overall 1‐week test‐retest correlations were high for both the RewP and FN and increased as a function of the number of trials, stabilizing after roughly 15 trials.…”

Section: Discussionsupporting

confidence: 89%

“…At Time 2, acceptable internal consistency for the RewP (14 trials) and FN (20 trials) required slightly more trials (Figure 3; final values based on the 53 participants without missing data). 2 Similarly, at Time 1, minimum recommended dependability scores of 0.7 or above (Clayson & Miller, 2016b) were reached for FN and RewP after 10 trials, while at Time 2, this threshold was reached for FN after 19 trials and for RewP after 15 trials. DRewP showed lower internal consistency than either of the two constituent measures as assessed using adjusted a (Furr & Bacharach, 2013).…”

Section: Cross-sectional Psychometrics Of Rewp and Fnmentioning

confidence: 98%

See 1 more Smart Citation

Reliability of the electrocortical response to gains and losses in the doors task

et al. 2017

View full text Add to dashboard Cite

The ability to differentiate between rewards and losses is critical for motivated action, and aberrant reward and loss processing has been associated with psychopathology. The reward positivity (RewP) and feedback negativity (FN) are ERPs elicited by monetary gains and losses, respectively, and are promising individual difference measures. However, few studies have reported on the psychometric properties of the RewP and FN-crucial characteristics necessary for valid individual difference measures. The current study examined the internal consistency and 1-week test-retest reliability of the RewP and FN as elicited by the doors task among 59 young adults. The RewP, FN, and their difference score (ΔRewP) all showed significant correlations between Time 1 and Time 2. The RewP and FN also achieved acceptable internal consistency at both time points within 20 trials using both Cronbach's α and a generalizability theory-derived dependability measure. Internal consistency for ΔRewP was notably weaker at both time points, which is expected from two highly intercorrelated constituent scores. In conclusion, the RewP and FN have strong psychometric properties in a healthy adult sample. Future research is needed to assess the psychometric properties of these ERPs in different age cohorts and in clinical populations.

show abstract

Section: Discussionsupporting

confidence: 89%

Section: Cross-sectional Psychometrics Of Rewp and Fnmentioning

confidence: 98%

Reliability of the electrocortical response to gains and losses in the doors task

et al. 2017

View full text Add to dashboard Cite

show abstract

“…One of these is generalizability (g) theory, which accounts for variability in assessment conditions (e.g., setting, time, items, raters) that can affect measurements (Clayson & Miller, in press). Another is item-response theory, in which reliability is estimated for varying levels of a score continuum, rather than for a ‘test’ (measured attribute) as a whole.…”

Section: Discussionmentioning

confidence: 99%

Methodological issues in the use of individual brain measures to index trait liabilities: The example of noise-probe P3

Perkins

Yancey

Drislane

et al. 2017

International Journal of Psychophysiology

View full text Add to dashboard Cite

Recent research initiatives have called for an increased use of biological concepts and measures in defining and studying mental health problems, but important measurement-related challenges confront efforts in this direction. This article highlights some of these challenges with reference to an intriguing measure of neural reactivity: the probe P3 response, a mid-latency brain potential evoked by an intense, unexpected acoustic-probe stimulus. Using data for a large adult sample (N = 418), we report evidence that amplitude of probe P3 response to unwarned noise bursts occurring in a picture-viewing task exhibits robust, independent associations with two distinct trait constructs: weak inhibitory control (or disinhibition; DIS) and threat sensitivity (THT). Additionally, we report a selective association for THT with attentional suppression of probe P3 response during viewing of aversive pictures compared to neutral. These results point to separable elements of variance underlying the probe P3 response, including one element reflecting DIS-related variations in cognitive-elaborative processing, and others reflecting THT-related variations in aversive foreground engagement and abrupt defensive reorientation. Key measurement issues are considered in relation to these specific findings, and methodological and statistical approaches for addressing these issues are discussed in relation to advancement of a quantitatively sound, biologically informed science of psychopathology.

show abstract

“…Given that statistical power was low for studies of smaller effects and that most studies used more complicated statistical analyses, many observed statistical effect sizes are likely exaggerated due to the statistical significance threshold commonly applied to published studies (Gelman, 2018;Rosenthal, 1979;Simmons et al, 2011). This bias to publish statistically significant effects incentivizes studying small samples and noisy measurements, because researchers can exploit the garden of forking paths (or researcher degrees of freedom) to find statistically significant effects (Baldwin, 2017;Brand & Bradley, 2016;Clayson & Miller, 2017b;Gelman, 2018;Gelman & Carlin, 2014;Larson & Carbine, 2017;Loken & Gelman, 2017).…”

Section: Sample Size and Statistical Powermentioning

confidence: 99%

Methodological reporting behavior, sample sizes, and statistical power in studies of event‐related potentials: Barriers to reproducibility and replicability

et al. 2019

Self Cite

View full text Add to dashboard Cite

Methodological reporting guidelines for studies of ERPs were updated in Psychophysiology in 2014. These guidelines facilitate the communication of key methodological parameters (e.g., preprocessing steps). Failing to report key parameters represents a barrier to replication efforts, and difficulty with replicability increases in the presence of small sample sizes and low statistical power. We assessed whether guidelines are followed and estimated the average sample size and power in recent research. Reporting behavior, sample sizes, and statistical designs were coded for 150 randomly sampled articles from five high‐impact journals that frequently published ERP research from 2011 to 2017. An average of 63% of guidelines were reported, and reporting behavior was similar across journals, suggesting that gaps in reporting is a shortcoming of the field rather than any specific journal. Publication of the guidelines article had no impact on reporting behavior, suggesting that editors and peer reviewers are not enforcing these recommendations. The average sample size per group was 21. Statistical power was conservatively estimated as .72‒.98 for a large effect size, .35‒.73 for a medium effect, and .10‒.18 for a small effect. These findings indicate that failing to report key guidelines is ubiquitous and that ERP studies are primarily powered to detect large effects. Such low power and insufficient following of reporting guidelines represent substantial barriers to replication efforts. The methodological transparency and replicability of studies can be improved by the open sharing of processing code and experimental tasks and by a priori sample size calculations to ensure adequately powered studies.

show abstract

Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting

Cited by 117 publications

References 93 publications

Reliability of the electrocortical response to gains and losses in the doors task

Reliability of the electrocortical response to gains and losses in the doors task

Methodological issues in the use of individual brain measures to index trait liabilities: The example of noise-probe P3

Methodological reporting behavior, sample sizes, and statistical power in studies of event‐related potentials: Barriers to reproducibility and replicability

Contact Info

Product

Resources

About