The “Reading the Mind in the Eyes” Test Shows Poor Psychometric Properties in a Large, Demographically Representative U.S. Sample

Abstract

The Reading the Mind in the Eyes test (RMET) is a widely used measure of theory of mind (ToM). Despite its popularity, there are questions regarding the RMET’s psychometric properties. In the current study, we examined the RMET in a representative U.S. sample of 1,181 adults. Key analyses included conducting an exploratory factor analysis on the full sample and examining whether there is a different factor structure in individuals with high versus low scores on the 28-item autism spectrum quotient (AQ-28). We identified overlapping, but distinct, three-factor models for the full sample and the two subgroups. In all cases, each of the three models showed inadequate model fit. We also found other limitations of the RMET, including that nearly a quarter of the RMET items did not meet the criteria for inclusion in the RMET that were established in the original validation study. Due to the RMET’s weak psychometric properties and the uncertain validity of individual items, as indicated by our study and previous studies, we conclude that significant caution is warranted when using the RMET as a measure of ToM.

Social cognitive disruptions in Multiple Sclerosis: The role of executive (dys)function.

“…However, such crude metrics can only offer limited insights into each of these seemingly complex cognitive operations. This is true especially when examining accuracy across all items of the RMET; recent meta-analyses have shown this task to have a multidimensional structure, in which subsets of items appear to assess different aspects of social cognition (Higgins et al, 2022). Similarly, responses to subsets of items on the FPT task can be combined to assess different dimensions of social awareness (e.g., understanding others' intentions, empathic awareness).…”

“…We measured emotion perception with the Reading the Mind in the Eyes Test (Baron-Cohen et al, 2001), given meta-analytic evidence of reliable performance deficits on this task in individuals with MS (see Bora et al, 2016;Cotter et al, 2016;Lin et al, 2021). Although the RMET has been used extensively as a measure of ToM in studies investigating social cognition into MS, formal assessments of its factorial structure, construct validity and associations with other tasks suggest that it more likely measures the accuracy with which emotions are perceived (Higgins et al, 2022;Kittel et al, 2022;Oakley et al, 2016;Quesque & Rossetti, 2020;Schurz et al, 2021). To measure ToM, we utilised another tool employed commonly in this area of research -the Faux Pas Test (Gregory et al, 2002).…”

Confirmatory factor analysis of the Brief Reading the Mind in the Eyes Test (RMET-10) in three large samples reveals strong evidence of unidimensionality

“…States residents (for further details, see Higgins et al, 2022). The 10 items from the RMET-10 Olderbak et al, 2015) were administered in all three samples.…”

“…In light of its extensive usage as an operationalization of cognitive empathy, particularly in clinical settings (e.g., Chander et al, 2020), the psychometric properties of the RMET have rightly received a great deal of attention. Despite the RMET's widespread use, however, the extant psychometric literature indicates that the full 36-item Eyes Test has poor internal consistency and test-retest reliability, alongside poor factorial validity and convergent validity (Harkness et al, 2010;Olderbak et al, 2015;Voracek & Dressler, 2006;Higgins, Ross, Langdon, & Polito, 2022). This has resulted in various research efforts to improve upon the RMET's psychometric properties.…”

Social cognitive disruptions in multiple sclerosis: The role of executive (dys)function.

“…We measured emotion perception with the reading the mind in the eyes test (RMET; Baron-Cohen et al, 2001), given meta-analytic evidence of reliable performance deficits on this task in individuals with MS (see Bora et al, 2016; Cotter et al, 2016; Lin et al, 2021). Although the RMET has been used extensively as a measure of ToM in studies investigating social cognition into MS, formal assessments of its factorial structure, construct validity and associations with other tasks suggest that it more likely measures the accuracy with which emotions are perceived (Higgins et al, 2022; Kittel et al, 2022; Oakley et al, 2016; Quesque & Rossetti, 2020; Schurz et al, 2021). To measure ToM, we utilized another tool employed commonly in this area of research—the faux pas test (FPT; Gregory et al, 2002).…”