Searching for Prosociality in Qualitative Data: Comparing Manual, Closed–Vocabulary, and Open–Vocabulary Methods

Abstract

Although most people present themselves as possessing prosocial traits, people differ in the extent to which they actually act prosocially in everyday life. Qualitative data that were not ostensibly collected to measure prosociality might contain information about prosocial dispositions that is not distorted by self‐presentation concerns. This paper seeks to characterise charitable donors from qualitative data. We compared a manual approach of extracting predictors from participants' self‐described personal strivings to two automated approaches: A summation of words predefined as prosocial and a support vector machine classifier. Although variables extracted by the support vector machine predicted donation behaviour well in the training sample (N = 984), virtually, no variables from any method significantly predicted donations in a holdout sample (N = 496). Raters' attempts to predict donations to charity based on reading participants' personal strivings were also unsuccessful. However, raters' predictions were associated with past charitable involvement. In sum, predictors derived from personal strivings did not robustly explain variation in charitable behaviour, but personal strivings may nevertheless contain some information about trait prosociality. The sparseness of personal strivings data, rather than the irrelevance of open‐ended text or individual differences in goal pursuit, likely explains their limited value in predicting prosocial behaviour. © 2020 European Association of Personality Psychology

Beyond English: Considering Language and Culture in Psychological Text Analysis

“…Compared to that, open-vocabulary approaches operate from "bottom-up" (data-driven), that is, based on language (text) as such. Algorithms identify related clusters of units (lexical units or elements, for example, punctuation) that naturally occur (and co-occur) within a large set of texts and find lexical and semantic patterns that appear (and appear together) in the data (Park et al, 2015;McAuliffe et al, 2020). Both approaches have their pros and cons; as stated by Eichstaedt et al, "Closed-vocabulary approaches can be rigid, while open-vocabulary approaches can be sensitive to idiosyncrasies of the dataset and the modeler's choices about parameters.…”

Understanding Ordinary Goal Pursuit: Describing and Predicting Success in New Year’s Resolutions

“…Prior studies of New Year's resolutions measured resolutions with broad selfreport nomothetic categories rather than open-ended text (e.g., Woolley & Fishbach, 2016). However, prior studies of mid-level goals, and in particular people's personal strivings, have used idiographic approaches combined with nomothetic coding based on existing manuals or by bottom-up categories, derived qualitatively or using automated methods (McAuliffe et al, 2020;Veilleux et al, 2018).…”

The kernel of truth in text-based personality assessment: A meta-analysis of the relations between the Big Five and the Linguistic Inquiry and Word Count (LIWC).

“…10Note, however, that LIWC is based on a straightforward counting method. This means that it is unqualified to understand the context and subtle nuances of language (e.g., irony, humor) like human judges do (McAuliffe et al, 2020; Pang & Ring, 2020), it does not analyze text using sophisticated machine learning approaches like the IBM-WPI does (IBM, 2021), it is not based on personality lexical studies like the HTTP (Holtrop et al, 2022), neither was it developed to measure personality in the first place.…”