A Deep Learning Perspective on Beauty, Sentiment, and Remembrance of Art

Abstract

With the emergence of large digitized fine art collections and the successful performance of deep learning techniques, new research prospects unfold in the intersection of artificial intelligence and art. In order to explore the applicability of deep learning techniques in understanding art images beyond object recognition and classification, we employ convolutional neural networks (CNN) to predict scores related to three subjective aspects of human perception: aesthetic evaluation of the image, sentiment evoked by the image, and memorability of the image. For each concept, we evaluate several different CNN models trained on various natural image datasets and select the best performing model based on the qualitative results and the comparison with existing subjective ratings of artworks. Furthermore, we employ different decision treebased machine learning models to analyze the relative importance of various image features related to the content, composition, and color in determining image aesthetics, visual sentiment, and memorability scores. Our findings suggest that content and image lighting have significant influence on aesthetics, in which color vividness and harmony strongly influence sentiment prediction, while object emphasis has a high impact on memorability. In addition, we explore the predicted aesthetic, sentiment, and memorability scores in the context of art history by analyzing their distribution in regard to different artistic styles, genres, artists, and centuries. The presented approach enables new ways of exploring fine art collections based on highly subjective aspects of art, as well as represents one step forward toward bridging the gap between traditional formal analysis and the computational analysis of fine art. INDEX TERMS Convolutional neural networks, image aesthetics, image memorability, fine art, visual sentiment.

Construction of a Soundscape-Based Media Art Exhibition to Improve User Appreciation Experience by Using Deep Neural Networks

“…Third, we discuss how a deep neural network was able to recommend music and paintings in the above consideration. The study by [77] dealt with the perspectives of beauty, sentiment, and remembrance of art in deep neural networks. The study by [77] quantitatively and qualitatively analyzed three subjective aspects of human consciousness: image features in relation to aesthetics, sentiment, and memorability.…”

An Information Theory Approach to Aesthetic Assessment of Visual Patterns

“…The meaning and the deeper relation of the link between information theory and aesthetic are to be further investigated in future work. Finding the most fundamental law or the optimization process that underlies aesthetically appealing patterns would be of great interest for the research in this area and for many applications [ 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 ]. It is interesting to see whether the proposed approach has any link to the aesthetic judgment mechanism in the brain, and how is that related to information theory.…”

ColorEmo: Color Hunt with Affective Words

“…Cohen-Or et al applied the harmonic models for image color harmonization [28]. Classic color harmony theory combined with a new deep learning method is used to evaluate the beauty, sentiment, and remembrance of art [29].…”