A Comparison of the Sensitivity of Four Indirect Evaluation Measures to Evaluative Information

Abstract

Indirect evaluation measures are used as a dependent measure to assess the impact of experimental interventions on shifting pre-existing attitudes or creating new attitudes. In four experiments (total N = 13,894), we compared the sensitivity of four indirect evaluation measures to evaluative information about two novel targets. Evaluative sensitivity was strongest for the Implicit Association Test (IAT). Other measures were more similar in their sensitivity, but the pattern, from stronger to the weakest was the Affect Misattribution Procedure (AMP), the Sorting Paired Features (SPF), and then Evaluative Priming task (EPT). To the extent that these findings are generalizable to related research applications, these results suggest that the measures differ in their research efficiency (power). For example, to achieve 80% power to detect the evaluative learning effect in the present studies, direct self-report would require 10 participants, the IAT 28 participants, the AMP 57, the SPF 79, and the EPT 131.

Understanding Implicit Bias: Putting the Criticism into Perspective

“…Payne and Lundberg 2014). Ultimately, the relatively low correlations between implicit measures are not so much an anomaly that threatens the field as it is a pedestrian empirical finding, which has begun to be explored (e.g., Bar-Anan and Nosek 2016;Moran et al 2017). We would certainly welcome further theory-based and experimental investigation into the mechanisms explaining performance on these distinct measures (cf.…”

“…Payne and Lundberg ). Ultimately, the relatively low correlations between implicit measures are not so much an anomaly that threatens the field as it is a pedestrian empirical finding, which has begun to be explored (e.g., Bar‐Anan and Nosek ; Moran et al . ).…”