Four dimensions characterize attributions from faces using a representative set of English trait words

Abstract

People readily (but often inaccurately) attribute traits to others based on faces. While the details of attributions depend on the language available to describe social traits, psychological theories argue that two or three dimensions (such as valence and dominance) summarize social trait attributions from faces. However, prior work has used only a small number of trait words (12 to 18), limiting conclusions to date. In two large-scale, preregistered studies we ask participants to rate 100 faces (obtained from existing face stimuli sets), using a list of 100 English trait words that we derived using deep neural network analysis of words that have been used by other participants in prior studies to describe faces. In study 1 we find that these attributions are best described by four psychological dimensions, which we interpret as “warmth”, “competence”, “femininity”, and “youth”. In study 2 we partially reproduce these four dimensions using the same stimuli among additional participant raters from multiple regions around the world, in both aggregated and individual-level data. These results provide a comprehensive characterization of trait attributions from faces, although we note our conclusions are limited by the scope of our study (in particular we note only white faces and English trait words were included).

A Cautionary Note on Predicting Social Judgments from Faces with Deep Neural Networks

“…The models were tested on five out-of-sample independent datasets that are publicly available (Lin et al, 2021;Oh et al, 2020;Oosterhof & Todorov, 2008;Walker et al, 2018;White et al, 2017). These test datasets were selected to sample social judgments from different types of faces, including studio portraits of frontal, neutral faces, computergenerated faces, and ambient photos of faces taken under unconstrained conditions.…”

“…All faces in our training and test datasets were limited to white faces; the effects of race and context (e.g., image background and facial expression) are beyond the scope of our current study. Specifically, the Lin et al (2021) dataset included ratings for 100 studio portraits of frontal, neutral, white faces (of which 60 were nonoverlapping with the training dataset, i.e., 60 novel faces) on 100 social attributes. The Oh et al (2020) dataset included ratings for 66 novel studio portraits of frontal, neutral, white faces on 14 social attributes.…”

“…Therefore, in the case where the same social attribute in the training dataset was not available in the test dataset, we used the synonym/antonym of the fitted social attribute in the test dataset (if available). Based on this rationale, we tested the models that were fit to the corresponding social attributes in the Chicago Face Database on nine social attributes in the Lin et al (2021) dataset, four social attributes in the Oh et al (2020) dataset, four social attributes in the Oosterhof and Todorov (2008) dataset, and three social attributes in the White et al (2017) dataset.…”

“…It remains an open question how well this approach generalizes out-of-sample, both across face databases and across human raters, which is a growing concern in modern machine learning for practical applications (D'Amour et al, 2020). Third, recent findings show that social judgments from faces made by human participants on a large number of social attributes can be captured by only a small number (two to four) of psychological dimensions (Lin et al, 2021;Oosterhof & Todorov, 2008;Sutherland et al, 2013). These findings suggest that many social judgments from faces are highly correlated, raising the possibility that models trained to predict one social judgment may also predict other social judgments.…”

“…In the real world, these judgments can show large individual differences and context effects: not only are they invalid, but consensus can also be difficult to achieve even for stimuli argued to be universal, such as emotional facial expressions (Barrett et al, 2019). With these constraints in mind, it remains a fact that people make many judgments about other people solely from faces in the absence of context or other information (e.g., deciding not to date someone just based on profile photos on dating sites), and the underlying psychological dimensions that explain the most variance in these judgments show considerable consensus across cultures (Lin et al, 2021).…”