Empirical recommendations for improving the stability of the dot-probe task in clinical research.

Abstract

The dot-probe task has been widely used in research to produce an index of biased attention based on reaction times (RTs). Despite its popularity, very few published studies have examined psychometric properties of the task, including test-retest reliability, and no previous study has examined reliability in clinically anxious samples or systematically explored the effects of task design and analysis decisions on reliability. In the current analysis, we used dot-probe data from 3 studies in which attention bias toward threat-related faces was assessed at multiple (Ն5) time-points. Two of the studies were similar (adults with social anxiety disorder, similar design features) whereas 1 was more disparate (pediatric healthy volunteers, distinct task design). We explored the effects of analysis choices (e.g., bias score formula, outlier handling method) on reliability and searched for convergent findings across the 3 studies. We found that, when concurrently considering the 3 studies, the most reliable RT index of bias used data from dot-bottom trials, comparing congruent to incongruent trials, with rescaled outliers, particularly after averaging across more than 1 assessment point. Although reliability of RT bias indices was moderate to low, within-session variability in bias (attention bias variability; ABV), a recently proposed RT index, was more reliable across sessions. Several eyetracking-based indices of attention bias (available in the pediatric healthy sample only) showed reliability that matched the optimal RT index (ABV). On the basis of these findings, we make specific recommendations to researchers using the dot-probe, particularly those wishing to investigate individual differences and/or single-patient applications.

Gender Difference in Attentional Bias Toward Negative and Positive Stimuli in Generalized Anxiety Disorder

“…Researchers have proposed measures of attentional bias (e.g., Compton, 2000;Iacoviello et al, 2014;Koster et al, 2006;Zvielli, Bernstein, & Koster, 2014) with differing psychometric properties (e.g., Iacoviello et al, 2014;Price et al, 2014). In order to determine the psychometric properties of our measure of bias we conducted split-half reliability analysis (Nunnally, 1978).…”

“…To examine this three-way interaction further, we computed a bias score. However, recent research suggests investigators should examine the psychometric properties of their bias score (e.g., Price et al, 2014).…”

Attention bias modification for social anxiety: A systematic review and meta-analysis

“…On the other hand, extant procedures for assessing AB by using reaction-time indices are insufficiently reliable to detect changes in AB (e.g., Dear, Sharpe, Nicholas, & Refshauge, 2011;Schmukle, 2005;Staugaard, 2009;Waechter, Nelson, Wright, Hyatt, & Oakman, 2014). New reaction-time methods for quantifying AB have emerged (Price et al, 2015;Zvielli, Bernstein, & Koster, in press) that conceptualize AB as a dynamic process rather than a static trait. Preliminary research involving these new methods of measuring AB yield modest, but larger, internal reliability estimates vis-à-vis traditional indices of AB that yield near-zero estimates (Zvielli et al, in press).…”

Combined effects of cognitive bias for food cues and poor inhibitory control on unhealthy food intake

“…The finding for attentional bias is not surprising as previous research has been inconsistent, and some studies have shown no link with consumption (Field et al, 2007;Fadardi & Cox, 2009;Field et al, 2009;Hardman et al, 2013). Questions have also been raised regarding the reliability of the dot probe task (Schmulke, 2005;Price et al, 2014).…”