Audiovisual integration in social evaluation.

Abstract

Our social evaluation of other people is influenced by their faces and their voices. However, rather little is known about how these channels combine in forming 'first impressions'. Over five experiments we investigate the relative contributions of facial and vocal information for social judgements: dominance and trustworthiness. The experiments manipulate each of these sources of information within-person, combining faces and voices giving rise to different social attributions. We report that vocal pitch is a reliable source of information for judgements of dominance (Study 1) but not trustworthiness (Study 4). Faces and voices make reliable, but independent contributions to social evaluation. However, voices have the larger influence in judgements of dominance (Study 2), whereas faces have the larger influence in judgements of trustworthiness (Study 5). The independent contribution of the two sources appears to be mandatory, as instructions to ignore one channel do not eliminate its influence (Study 3). Our results show that information contained in both the face and the voice contributes to first impression formation. This combination is, to some degree, outside conscious control, and the weighting of channel contribution varies according the trait being perceived.

The Role of Face and Voice Cues in Predicting the Outcome of Student Representative Elections

“…Our findings on the importance of face cues for election decisions are in line with those of Klofstad (2017) as well as with previous findings on the greater contribution of face cues when judging trustworthiness (Mileva et al, 2018;Tsankova et al, 2015). It should, however, be noted that trustworthiness as judged from the face and from the voice were very highly correlated, suggesting that first impressions from faces and voices both signal the same integrated person evaluation.…”

Judgements of a speaker’s personality are correlated across differing content and stimulus type

“…Further, in voice research, this cross-participant consistency has been established within given specific durations of vocalisations or utterances; high inter-rater reliability for ratings has been found using sub-second utterances of vowels or words [ 1 , 22 , 46 , 53 , 54 ], as well as from longer sentences and passages [ 6 , 7 , 27 , 29 , 30 , 47 ]. For illustration, McAleer et al [ 1 ] reported very high Cronbach’s Alpha for ratings towards voices across a number of personality traits (all alpha’s > .88) which is in line with the high inter-rater reliability found in similar face perception studies (all alphas > .9 in [ 38 ]; = .98 in [ 39 ]; > .7 in [ 41 , 42 ]; > .86 in [ 55 ]) though with some variation depending on traits (e.g. attractiveness: .95 - .97, trustworthiness: .92, aggressiveness: .75 - .89 in [ 40 ]).…”

“…It is possible that this difference was weakest in dominance as previous literature has shown this trait to be driven by more stable voice metrics, such as formant and HNR, whereas trust and attractiveness may be more related to pitch [ 1 , 49 , 70 , 73 ]. Also, audio-visual integration research suggests that dominance is more driven by the voice, whereas trustworthiness and attractiveness appear driven either by the face or the integration of modalities [ 46 , 55 ]. Thus, perceived dominance in voices may be so prevalent that it does not matter whether you hear one word or one sentence.…”

The sound of trustworthiness: Acoustic-based modulation of perceived voice personality

“…For voices, significant correlations between personality ratings and acoustical parameters such as average fundamental frequency (f0) have been observed [2, 4–6, 11, 12] suggesting an acoustical basis for voice personality judgments by which principled changes in voice acoustics could potentially map onto desired changes in perceived personality. But until now the correlational approach has fallen short of precisely characterizing this acoustical basis.…”