"Wow!" and "Aah": Bimodal Stimuli of Affective States with High and Low Intensity Displays

Abstract

Our study focused on the ability of humans to correctly rate the valence of human facial expressions and vocalizations of high (pain and pleasure) and low intensity (laugh and neutral expression/speech) affective states. The study was conducted online and used a large sample (n=902) of respondents. The task was to categorize whether the human vocalization and facial expression as positive, neutral, or negative. The stimuli were audio records and pictures of facial expressions extracted from freely downloadable online videos and can be considered semi-naturalistic. Each rating participant was presented simultaneously with the facial expression and the vocalization of the affective states. Two of these with high intensity (pain and pleasure) and two of low intensity (laugh/smile and neutral). Each affective state was expressed and rated by female and male expressers. Using a Bayesian statistical approach, we could test due-to-chance probabilities (guessing). The outcomes support the prediction that affective states with higher intensity are harder to correctly rate, even with a bimodal presentation. Furthermore, in comparison with previous unimodal studies using the same stimuli, the results provide novel insight into systematic perception. Adding the vocalizations to the facial expressions results in the participants being more convinced in their choices independently of the correctness of the assessment as previously shown in auditory stimuli perception.