Visual Analysis of Emotions Using AI Image-Processing Software: Possible Male/Female Differences between the Emotion Pairs “Neutral”–“Fear” and “Pleasure”–“Pain”

Prossinger, Hermann; Hladký, Tomáš; Binter, Jakub; Boschetti, Silvia; Říha, Daniel

doi:10.1145/3453892.3461656

Cited by 7 publications

(10 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An interesting comparison, using the same stimuli as in this paper, is provided in another study (Prossinger et al 2021b); it evaluated the precision of distinguishing stimuli categories.…”

Section: Human Ratings Versus Ai Ratingsmentioning

confidence: 99%

“Eye can’t see the difference”: Facial Expressions of Pain, Pleasure, and Fear Are Consistently Rated Due to Chance

Boschetti¹,

Prossinger²,

Hladký³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Our research consisted of two studies focusing on the probability of humans being able to perceive the difference between faces expressing pain versus pleasure. As controls, we included: smile, neutral facial expression, and expression of fear. The first study was online and was conducted using a large sample (n=902) of respondents. The second study was conducted in a laboratory setting and involved a stress induction procedure. For both, the task was to categorize whether the facial expression was rated positive, neutral or negative. Stimuli were faces extracted from freely downloadable online videos. Each rating participant (rater) was presented with five facial expressions (stimuli) of five females and five males. All raters were presented with the stimuli twice so as to evaluate the consistency of the ratings. Beforehand, we tested for stimuli differences using specialized software and found decisive differences. Using a Bayesian statistical approach, we could test for consistencies and due-to-chance probabilities. The results support the expectation that the results are not repeatable but indeed solely due to chance, diminishing the communication value of the expressions of pain and pleasure. The expression of fear was also rated due to chance, but not neutral and smile. Stress induction did have an impact on the perception of pleasure.

show abstract

“…An interesting comparison, using the same stimuli as in this paper, is provided in another study (Prossinger et al 2021b); it evaluated the precision of distinguishing stimuli categories.…”

Section: Human Ratings Versus Ai Ratingsmentioning

confidence: 99%

“Eye can’t see the difference”: Facial Expressions of Pain, Pleasure, and Fear Are Consistently Rated Due to Chance

Boschetti¹,

Prossinger²,

Hladký³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…AI facial analysis should be able to identify such differences, thus bridging this gap and implying that it is superior to human vision in this regard. Already in our pilot study [1], this distinguishability was found in women but not in men. Based on these published outcomes, we increased the sample size but restricted the study to actresses only.…”

Section: Previous Reasearch Into Facial Expression Of (Intense) Affec...mentioning

confidence: 64%

“…This manuscript reports results that are extensions of the pilot study published in 2021 [1]. There, we investigated, using AI, whether it is possible to distinguish the facial expression of pain from that of pleasure in women.…”

Section: Introductionmentioning

confidence: 86%

“…There exists a considerable amount of literature [2][3][4][5][6] dealing with the recognition and identification of affective states putatively communicated by facial expressions. Novel in the approach we present here are the following: (1) to use one AI tool to extract faces from video frames; (2) to use another AI tool (an autoencoder) to determine their dimensionreduced feature vectors; (3) to quantify their differences (if any); and (4) to implement the machinery of multivariate statistical analysis, combined with clustering algorithms, to detect noise contributions and identify patterns.…”

Section: Using Ai As a Novel Approach To Analyzing Facial Expressions...mentioning

confidence: 99%

See 1 more Smart Citation

Determination of “Neutral”–“Pain”, “Neutral”–“Pleasure”, and “Pleasure”–“Pain” Affective State Distances by Using AI Image Analysis of Facial Expressions

et al. 2022

Self Cite

View full text Add to dashboard Cite

(1) Background: In addition to verbalizations, facial expressions advertise one’s affective state. There is an ongoing debate concerning the communicative value of the facial expressions of pain and of pleasure, and to what extent humans can distinguish between these. We introduce a novel method of analysis by replacing human ratings with outputs from image analysis software. (2) Methods: We use image analysis software to extract feature vectors of the facial expressions neutral, pain, and pleasure displayed by 20 actresses. We dimension-reduced these feature vectors, used singular value decomposition to eliminate noise, and then used hierarchical agglomerative clustering to detect patterns. (3) Results: The vector norms for pain–pleasure were rarely less than the distances pain–neutral and pleasure–neutral. The pain–pleasure distances were Weibull-distributed and noise contributed 10% to the signal. The noise-free distances clustered in four clusters and two isolates. (4) Conclusions: AI methods of image recognition are superior to human abilities in distinguishing between facial expressions of pain and pleasure. Statistical methods and hierarchical clustering offer possible explanations as to why humans fail. The reliability of commercial software, which attempts to identify facial expressions of affective states, can be improved by using the results of our analyses.

show abstract

“…A method for obtaining the stimuli, as well as their use were presented in a previously published article (Prossinger, 2021b).…”

Section: Stimulus Creationmentioning

confidence: 99%

“Eye can’t see the difference”: Facial Expressions of Pain, Pleasure, and Fear Are Consistently Rated Due to Chance

Boschetti¹,

Prossinger²,

Hladký³

et al. 2022

View full text Add to dashboard Cite

Our research consisted of two studies focusing on the probability of humans being able to perceive the difference between faces expressing pain versus pleasure. As controls, we included: smile, neutral facial expression, and expression of fear. The first study was conducted online and used a large sample (n = 902) of respondents. The second study was conducted in a laboratory setting and involved a stress induction procedure. For both, the task was to categorize whether the facial expression was rated positive, neutral or negative. Stimuli were faces extracted from freely downloadable online videos. Each rating participant (rater) was presented with five facial expressions (stimuli) of five females and of five males. All raters were presented with the stimuli twice so as to evaluate the consistency of the ratings. Beforehand, we tested for stimuli differences using specialized software and found decisive differences. Using a Bayesian statistical approach, we could test for consistencies and due-to-chance probabilities. The results support the prediction that the results are not repeatable but are solely due to chance, decreasing the communication value of the expressions of pain and pleasure. The expression of fear was also rated due to chance, but neither neutral nor smile. Stress induction did have an impact on the perception of pleasure.

show abstract

Visual Analysis of Emotions Using AI Image-Processing Software: Possible Male/Female Differences between the Emotion Pairs “Neutral”–“Fear” and “Pleasure”–“Pain”

Cited by 7 publications

References 3 publications

“Eye can’t see the difference”: Facial Expressions of Pain, Pleasure, and Fear Are Consistently Rated Due to Chance

“Eye can’t see the difference”: Facial Expressions of Pain, Pleasure, and Fear Are Consistently Rated Due to Chance

Determination of “Neutral”–“Pain”, “Neutral”–“Pleasure”, and “Pleasure”–“Pain” Affective State Distances by Using AI Image Analysis of Facial Expressions

“Eye can’t see the difference”: Facial Expressions of Pain, Pleasure, and Fear Are Consistently Rated Due to Chance

Contact Info

Product

Resources

About