Using game-based assessments (GBAs) to assess and select job applicants presents the dual challenges of measuring intended job-relevant constructs while analyzing GBA data that contain more predictors than observations. Exploring those challenges, we analyzed two GBAs that were designed to measure conscientiousness facets (i.e., achievement striving, self-discipline, and cautiousness). Scores on traditional measures of personality and cognitive ability were modeled using either a restricted set of GBA predictors using cross-validated ordinary least squares (OLS) regression or by the fuller set (p = 248) using random forests regression. Overall, the prediction of personality was near-zero; but the latter approach explained 14%-30% of the variance in predicting cognitive ability. Our findings warn of GBAs potentially measuring unintended constructs rather than their intended constructs.

This conceptual paper examines the promises and critical challenges posed by contemporary personality measurement using big data. More specifically, the paper provides (i) an introduction to the type of technologies that give rise to big data, (ii) an overview of how big data is used in personality research and how it might be used in the future, (iii) a framework for approaching big data in personality science, (iv) an exploration of ideas that connect psychometric reliability and validity, as well as principles of fairness and privacy, to measures of personality that use big data, (v) a discussion emphasizing the importance of collaboration with other disciplines for personality psychologists seeking to adopt big data methods, and finally, (vi) a list of practical considerations for researchers seeking to move forward with big data personality measurement and research. It is expected that this paper will provide insights, guidance, and inspiration that helps personality researchers navigate the challenges and opportunities posed by using big data methods in personality measurement. © 2020 European Association of Personality Psychology

The essence of measurement invariance (MI) analysis is to test the assumption that observed scores on a scale accurately reflect respondents’ standings on a measured construct. Based on exploratory structural equation modeling, the current study examines gender-based MI in two Big Five measures of personality: the Mini-IPIP, and the Big Five Inventory (BFI) facet scales. We report results for MI based on both model fit indices and a practical significance index that quantifies the extent of noninvariance (i.e., dMACS). From the latter, we partition the observed group mean differences in scale scores into construct-irrelevant group differences versus construct-relevant group differences. In measures of Agreeableness, Conscientiousness, Extraversion, and Neuroticism across instruments, results supported metric invariance but not scalar invariance. That said, findings of statistical noninvariance were generally small in terms of practical effects, although some notable variability in the effects was evident. Overall, the current results provide evidence regarding gender-based MI of Big Five personality measures that are more detailed than that provided in past work. More generally, this study also provides useful guidance for future researchers investigating both the statistical and practical significance of measurement invariance.

No abstract