Test-Retest Reliability and Construct Validity of the Brief Dark Triad Measurements

Abstract

Despite the widespread use and popularity of the Dirty Dozen (DD) and Short Dark Triad (SD3) as measurements of “dark” personalities, there appears to be a lack of appropriately powered studies on the test-retest reliability (rtt) of the two brief measurements. We report 12-day test-retest of the DD and SD3 at the level of domains and items. Leveraging the data, we calculated the convergent and discriminant correlations of the DD and SD3 scales while controlling for measurement error, and also evaluated the reliability of items’ unique variances. Median rtts were 0.87 and 0.90 (N = 500) for the DD and SD3 scales respectively, substantially higher than their internal consistencies. Convergent correlations were 0.77, 0.63 and 0.64 for Machiavellianism, Narcissism and Psychopathy respectively, whereas discriminant correlations between the Machiavellianism and Psychopathy scales had a median of 0.65. The unique variances of all DD and SD3 items had significant and often very high rtts (medians 0.49 and 0.57, respectively). We emphasize the importance of rtt for scale development and validation and conclude that the DD and SD3 have hierarchical structures with a Dark Dyad (Narcissism and Psychopathy/Machiavellianism) at a higher level and a Dark Myriad (individual items) at a lower level.

Using retest-adjusted correlations as indicators of the semantic similarity of items.

“…There has been increasing interest in using retest correlations to adjust for measurement unreliability (e.g., Dragostinov & Mõttus, 2021; Mueller et al, 2015; Wood & Wortman, 2012; Wood et al, 2018). This suggestion has been based in part on recent evidence that retest correlations over intervals such as a couple weeks or months outperform more commonly employed reliability estimates toward predicting validity-related criteria that should be attenuated by unreliability, such as correlational estimates of self–other agreement or heritability (Henry et al, 2022; Lowman et al, 2018; McCrae et al, 2011; Mõttus et al, 2017).…”