Strong claims and weak evidence: Reassessing the predictive validity of the IAT.

Abstract

The authors reanalyzed data from 2 influential studies-A. R. McConnell and J. M. Leibold (2001) and J. C. Ziegert and P. J. Hanges (2005)-that explore links between implicit bias and discriminatory behavior and that have been invoked to support strong claims about the predictive validity of the Implicit Association Test. In both of these studies, the inclusion of race Implicit Association Test scores in regression models reduced prediction errors by only tiny amounts, and Implicit Association Test scores did not permit prediction of individual-level behaviors. Furthermore, the results were not robust when the impact of rater reliability, statistical specifications, and/or outliers were taken into account, and reanalysis of A. R. McConnell & J. M. Leibold (2001) revealed a pattern of behavior consistent with a pro-Black behavioral bias, rather than the anti-Black bias suggested in the original study.Keywords: Implicit Association Test, predictive validity, discrimination, implicit biasThe Implicit Association Test (IAT; Greenwald, McGhee & Schwartz, 1998) has become one of psychology's most popular exports to the wider social sciences and the law (e.g., Lane, Kang, & Banaji, 2007). The measure's popularity is easy to grasp: IAT researchers often inform test takers they harbor implicit or unconscious biases toward minorities that many test takers disavow at a conscious level (Dasgupta, Greenwald, & Banaji, 2003). According to the much-visited Project Implicit Web site, without constant vigilance these implicit biases may lead to unwanted behaviors:People who hold egalitarian conscious attitudes in the face of automatic White preferences may [be] able to function in nonprejudiced fashion partly by making active efforts to prevent their automatic White preference from producing discriminatory behavior. However, when they relax these active efforts, these nonprejudiced people may be likely to show discrimination in thought or behavior. (IAT Corporation, n.d., Question 16) It is this claimed connection between implicit attitudes and discrimination that can make IAT feedback particularly disturbing to test takers. It also is the feature that makes research on the IAT of broad interdisciplinary interest. If the race IAT reliably predicts discriminatory behavior that cannot be consciously controlled, then society should take note. As but one example, the great majority of White Americans who have taken the IAT have been classified as anti-Black. This then points to an epidemic, either of unconscious racism (Greenwald & Krieger, 2006) or of falsepositive accusations of unconscious racism (Mitchell & Tetlock, 2006).Given the importance of the link between IAT scores and behavior, one might expect to find a large body of data establishing this connection-indeed, Greenwald and Krieger (2006, p. 961) described the evidence that implicit bias leads to discriminatory behavior as "already substantial." In fact, researchers in only a few studies have examined the ability of the IAT to predict behavior of any type, and resear...

Weak criticisms and selective evidence: Reply to Blanton et al. (2009).

“…

We respond to a critique by H. Blanton et al (2009), challenging our previous work demonstrating that an Implicit Association Test designed to assess implicit prejudice reliably predicts intergroup discrimination (A. R. McConnell & J. M. Leibold, 2001). We outline 3 flawed aspects of the critique.

…”

“…We are pleased to offer our thoughts on the Blanton et al (2009) article. In July 2006, we were contacted by an author on that article (G. Mitchell, personal communication, July 25, 2006), who asked us for copies of the data sets for our publication, McConnell and Leibold (2001).…”

“…One of the points made by Blanton et al (2009) is that it is tenuous to make strong claims about relations between implicit attitudes and behavior based on a small amount of data. We agree that it would be preferable to see more studies demonstrate the implications of implicit attitudes for behavioral relations and, indeed, this was a primary motive behind our conducting the McConnell and Leibold (2001) study in the first place.…”

Consumption and climate change: Why we say one thing but do another in the face of our greatest threat

“…This is by no means universally accepted in the psychological literature; indeed it is currently the subject of much quite heated debate (see Blanton et al, 2006Blanton et al, , 2007Blanton et al, , 2009 for a critique of this position and Greenwald, Nosek and Sriram, 2006, McConnell and Leibold, 2009, and Ziegert and Hanges, 2009 for some rebuttals). However, it would seem that in some domains, this notion of implicit attitudes, deriving from various associative connections and operating unconsciously alongside our more reflective attitudes (and indeed conflicting with them on occasion), might have some credibility (Beattie, 2013: Kahneman, 2011.…”

Exploring the Reliability and Convergent Validity of Implicit Racial Evaluations

“…This study did not address the predictive validity of the IRAP, a crucial test of the value of any measure. Race IATs have been shown to predict laboratory examples of discriminatory behavior (e.g., Green, et al, 2007;Heider & Skowronski, 2007), although some recent meta-analyses have suggested that the predictive validity data as a whole is relatively unimpressive (Blanton, et al, 2009;Oswald, Mitchell, Blanton, Jaccard, & Tetlock, 2013). It seems important to subject the IRAP to an examination of predictive validity, given the divergence in methodology from the IAT and the more specificity of results it can provide.…”