Emotional valence categorization using holistic image features

Abstract

Can a machine learn to perceive emotions as evoked by an artwork? Here we propose an emotion categorization system, trained by ground truth from psychology studies. The training data contains emotional valences scored by human subjects on the International Affective Picture System (IAPS), a standard emotion evoking image set in psychology. Our approach is based on the assessment of local image statistics which are learned per emotional category using support vector machines. We show results for our system on the IAPS dataset, and for a collection of masterpieces. Although the results are preliminary, they demonstrate the potential of machines to elicit realistic emotions when considering masterpieces.

Affective Analysis of Abstract Paintings Using Statistical Analysis and Art Theory

“…The progress of computer vision research has become mature enough to predict emotions Machajdik & Hanbury [2010]; Yanulevskaya et al [2008], aesthetics Marchesotti et al [2011], and interestingness Isola et al [2011] of images, paintings, and even web pages Wu et al [2011]. Yanulevskaya et al [2008] proposed an emotion categorization system based on the assessment of local image statistics followed by supervised learning of emotion categories using Support Vector Machines.…”

Entity-Based Opinion Mining from Text and Multimedia

“…However, those published often use small datasets for their ground truth on which to build SVM classifiers. Evaluations show systems often respond only a little better than chance for trained emotions from general images [15]. The implication is that the feature selection for such classification is difficult.…”

“…State-of-the-art research on the sentiment analysis of images (see e.g. [15,45,46,16,47]) has already begun to explore how the analysis of textual content and the analysis of visual content can complement each other. Recently, we have been exploring how visual content and contextual information can be leveraged to train machines to predict facets related to opinion formation.…”

On shape and the computability of emotions

“…Previous work [11,26,18] predicted emotions aroused by images mainly through training classifiers on visual features to distinguish categorical emotions, such as happiness, anger, and sad.…”

“…However, the computational work on the IAPS dataset to understand the visual factors that affect emotions has been preliminary. Researchers [9,11,18,23,25,26] investigated factors such as color, texture, composition, and simple semantics to understand emotions, but have not quantitatively addressed the effect of perceptual shapes. The study that did explore shapes by Zhang et al [27] predicted emotions evoked by viewing abstract art images through low-level features like color, shape, and texture.…”