Deep Lexical Hypothesis: Identifying personality structure in natural language

Abstract

Recent advances in natural language processing (NLP) have produced general models that can perform complex tasks such as summarizing long passages and translating across languages. Here, we introduce a method to extract adjective similarities from language models as done with survey-based ratings in traditional psycholexical studies but using millions of times more text in a natural setting. The correlational structure produced through this method is highly similar to that of self- and other-ratings of 435 terms reported by Saucier and Goldberg (1996a). The first three unrotated factors produced using NLP are congruent with those in survey data, with coefficients of 0.89, 0.79, and 0.79. This structure is robust to many modeling decisions: adjective set, including those with 1,710 terms (Goldberg, 1982) and 18,000 terms (Allport & Odbert, 1936); the query used to extract correlations; and language model. Notably, Neuroticism and Openness are only weakly and inconsistently recovered. This is a new source of signal that is closer to the original (semantic) vision of the Lexical Hypothesis. The method can be applied where surveys cannot: in dozens of languages simultaneously, with tens of thousands of items, on historical text, and at extremely large scale for little cost. The code is made public to facilitate reproduction and fast iteration in new directions of research.

Frequency of use metrics for American English person descriptors: Extensions of Roivainen's internet search methodology

“…The primary contribution of this project is to make the books and search results available for researchers, especially personality psychologists, who seek information about the frequency of usage for person-descriptive terms. Most prominently, these data can contribute to the long arc of psycholexical research that began in earnest in 1936 (Allport & Odbert) and remains ongoing (Cutler & Condon, 2022). Prior work in this area has led to the identification of several multidimensional structural models of personality, including the Big Five (Goldberg, 1992), the HEXACO (Ashton et al, 2004) and the High Dimensional 20 (Saucier & Iurino, 2020).…”

“…The need to characterize longer lists of terms stems from the potential to use language models (aka natural language processing techniques) in personality structure research (Cutler & Condon, 2022;Jackson et al, 2021). These models include, for example, transformer-encoder models such as BERT (Devlin et al, 2019), DeBERTa (He et al, 2021), and GPT-3 (Brown et al, 2020).…”

“…Unlike traditional survey-based approaches that require ratings of each descriptor from individual raters, these models are not constrained by the attention and fatigue limits of human raters. This introduces the possibility of including many thousands of trait descriptive adjectives in analyses of personality structure (Cutler & Condon, 2022). In turn, this possibility increases the benefit of characterizing the full universe of person descriptors on various features, including the extent to which they are commonly used.…”

“…Specifically, the initial scope of the project included only the 2,818 descriptors reported in Condon, Coughlin, and Weston (2022). However, the results of a related project (Cutler & Condon, 2022) demonstrated that future research on personality structure need not be limited to relatively small sets of person-descriptive terms, and this prompted the decision to proceed with data collection for a much more comprehensive list, as described below.…”

Deep lexical hypothesis: Identifying personality structure in natural language.

“…Though the approaches clearly differ, NLP and survey-based methods are similar in that the underlying data are contributed by individual people who will influence the resulting structure in proportion to the total informational content of the full sample of contributors. Thurstone (1934) was developing, in essence, a type of language model when instructing respondents to rate themselves using an adjective checklist, as this data-generating process is functionally equivalent to the generation of a statement about one’s personality (Cutler, 2021; Cutler et al, 2021). A benefit of Thurstone’s approach is that the resulting data are highly structured, but the trade-off is substantial—the pool of respondents is limited to those who can be recruited and motivated to complete the checklist.…”