A Failed Cross-Validation Study on the Relationship between LIWC Linguistic Indicators and Personality: Exemplifying the Lack of Generalizability of Exploratory Studies

Abstract

(1) Background: Previous meta-analytic research found small to moderate relationships between the Big Five personality traits and different linguistic computational indicators. However, previous studies included multiple linguistic indicators to predict personality from an exploratory framework. The aim of this study was to conduct a cross-validation study analyzing the relationships between language indicators and personality traits to test the generalizability of previous results; (2) Methods: 643 Spanish undergraduate students were tasked to write a self-description in 500 words (which was evaluated with the LIWC) and to answer a standardized Big Five questionnaire. Two different analytical approaches using multiple linear regression were followed: first, using the complete data and, second, by conducting different cross-validation studies; (3) Results: The results showed medium effect sizes in the first analytical approach. On the contrary, it was found that language and personality relationships were not generalizable in the cross-validation studies; (4) Conclusions: We concluded that moderate effect sizes could be obtained when the language and personality relationships were analyzed in single samples, but it was not possible to generalize the model estimates to other samples. Thus, previous exploratory results found on this line of research appear to be incompatible with a nomothetic approach.

Exploring the Use of Contextual Analysis of Written Communications to Identify Fraud Risk: Analysis and Comparison of Enron and Facebook Internal Communications

“…Argamon and Levitan (2005) and Oberlander and Gill (2006) also used this approach and found that word usage could identify the trait of extroversion in writers. Martínez-Huertas, et al (2022) reported that LIWC variables had a significant relationship with big five personality traits (openness, conscientiousness, extraversion, agreeableness, and neuroticism) in their sample. Pennebaker and King (1999) and Pennebaker (2011) have written extensively about the relationship between the personality traits associated with writing style.…”

Editorial to the Special Issue “Feature Papers in Psychometrics and Educational Measurement”

“…This study, published in five volumes as Genetic Studies of Genius between 1926 and 1959, required the construction of a special instrument to accommodate Terman's subjects as adults, called the Concept Mastery Test [3]. This marked the beginning of experimental research on adults in the intellectual stratosphere using psychological techniques.…”

“…The article by Martínez-Huertas, Moreno, Olmos, Martínez-Mingo, and Jorge-Botana (2022) titled "A failed cross-validation study on the relationship between LIWC linguistic indicators and personality: Exemplifying the lack of generalizability of exploratory studies" [3] is a replication study to study the relationship between the "Big 5 personality traits" and linguistic indicators. In a sample of N = 643 Spanish undergraduate students, it was found that language and personality relationships were not generalizable in crossvalidation studies.…”