Reporting and interpreting effect sizes has been recommended by all major bodies within the field of Psychology. In this systematic review, we investigated the reporting of ES in six Social-Personality Psychology journals from 2018, given that this area has been at the center of Psychology’s replication crisis. Our results highlight that although ES reporting is near perfect (even for follow-up tests), interpreting the magnitude of ESs, including confidence intervals for ESs, and interpreting the precision of the confidence intervals needs development. We also highlight widespread confusion regarding the interpretations of the magnitude of effect sizes within the context of the research.

The Psychological Science Accelerator’s (PSA) primary project tested for latent structure using exploratory factor analysis and confirmatory factor analysis but we decided to diverge from this approach and model individual traits separately. Our interest mainly was in examining the interplay between “stimulus ethnicity” and “stimulus sex” to discover how differing levels of these criterion differ across region, country, lab etc. While the necessary and prerequisite hierarchical structural information about each trait could certainly be found within the primary project’s dataset, we did not assume that any specific factor structure from the PSA’s primary analysis would necessarily hold, therefore we based our decision to model the data from each trait separately using a mixed model framework.