Test-Retest Reliability of the Emotional Stroop Task: Examining the Paradox of Measurement Change

Abstract

The Emotional Stroop (ES) task (I. H. Gotlib & C. D. McCann, 1984) has been proposed as an experimental measure to assess the processing of emotion or the bias in attention of emotion-laden information. However, study results have not been consistent. To further examine its reliability for empirical research, the authors of this study administered the ES task to 33 participants on 2 separate occasions separated by 1 week. Results indicated that retest reliabilities for reaction times (RTs) derived from the 3 separate emotion conditions (manic, neutral, and depressive) across the 1 week interval were very high. However, consistent with previous research, the reliabilities were very low for the interference indices (manic and depressive). These low reliabilities reflect the very high intercorrelation between the RTs derived from the 3 conditions. The authors concluded that a better indicator of the reliability for this task is the individual RTs from each emotion condition.

Reliability of the emotional Stroop task: An investigation of patients with panic disorder

“…This pattern is in line with previous studies on the reliability of the emotional Stroop task in healthy subjects (Kindt et al, 1996;Siegrist, 1997;Eide et al, 2002;Strauss et al, 2005) which found low testeretest reliability scores for interference, but higher scores for response latency data. Those studies only reported testeretest reliability scores without investigating reliability for each single measurement.…”

“…Such a pattern of low reliability of the interference score (wÀ0.2 to 0.2) and high reliability of the response latencies (i.e. mean response latency emotional or neutral words; >0.80) has been replicated by more recent studies (Eide et al, 2002;Strauss et al, 2005). These reliability findings only apply to healthy subjects.…”

“…These reliability findings only apply to healthy subjects. However, the emotional Stroop task is frequently used in different clinical populations assuming stable interference effects and it is "important for researchers to determine how these interference reliability scores compare with scores in clinical populations, especially as the ES [emotional Stroop] task is considered an important tool in determining psychopathology in depressed and anxious individuals" (Eide et al, 2002, p. 519, see also Strauss et al, 2005, p. 336). From this perspective, it is necessary to evaluate the emotional Stroop task's reliability in patient samples, especially when emotional interference effects are supposed to reflect stable subject characteristics that might reflect symptom severity or might be changed by different therapeutic interventions.…”

Emotional Stroop task: effect of word arousal and subject anxiety on emotional interference

“…Interference scores of emotional stimuli from the Wrst half of each run were correlated with the interference scores from the second half. Split-half reliability amounted to 0.76 (Spearman-Brown corrected), which is above the reliability scores for emotional interference scores using a retest design with a 1-week interval (Eide et al 2002). Split-half reliability was 0.75 for negative and 0.65 for positive interference.…”

Getting Explicit About the Implicit

“…Slower reaction times when naming font color indicate that word content is more accessible at implicit levels. (Eide, Kemp, Silberstein, Nathan, & Stough, 2002;Kindt, Bierman, & Brosschot, 1996;Siegrist, 1997;Strauss, Allen, Jorgensen, & Cramer, 2005) Predictive validity Validated by several research teams and shown to predict outcomes including mental health problems (Manschreck et al, 1988;Rikers, Loyens, Winkel, Schmidt, & Sins, 2005) Validated by several research teams and shown to predict outcomes including task performance, organizational citizenship behavior, and counterproductive work behavior Johnson & Lord, 2010;Steele & Aronson, 1995) Validated by several research teams and shown to predict outcomes including expectations for unfair treatment and mental health problems (Ritter, Fischbein, & Lord, 2006;Williams, Mathews, & MacLeod, 1996) Correlations with explicit measures Low (Rudman & Borgida, 1995) Moderate Low (Egloff & Schmukle, 2004;Payne, Binik, Amsel, & Khalife, 2004) Vulnerability Personality, affect (for reviews of prior uses, see MacLeod, 1991;Williams et al, 1996) Mode of administration Internal consistency .55 to .94 Internal consistency .69 (continued) (Karpinski & Steinman, 2006) Correlations with explicit measures Varies from low to high by domain …”