Measuring photography aesthetics with deep CNNs

A Comprehensive Survey on Computational Aesthetic Evaluation of Visual Art Images: Metrics and Challenges

“…Beginning with the strong performance of Krizhevsky et al [18] in the image classification, the powerful feature representation learned with a growing amount of datasets, and feasible transfer learning [19] with fine-tuned Convolutional Neural Networks (CNN) [20], deep learning methods [21] have been applied to aesthetic quality assessment of visual art images, which can automatically learn effective aesthetic features from deep hidden layers to abstract image information without expert knowledge, thus showing outperformed evaluation capability than conventional handcrafted features. Researches in the aesthetic assessment of visual art images using deep learning approaches can be summarized into 4 major schemes (Figure 12), 1) Aesthetic scoring refers to Designs for effective loss functions and structures in output layers of CNNs [83,84] Distribution-oriented aesthetic representation [100] Context-aware attention-based module with distance loss function [85] Employ attributes as middlelevel representations to measure aesthetics [45,86] Jointly learn distributions of attributes and aesthetics [87,88] Employ CNN-LSTM architecture to produce comments on photo aesthetic aspects [46,47,49] Attentive LSTM for generating emotional explanations in artworks [53] different aesthetic quality levels (binary labels as "positive" or "negative"), or continuous images aesthetic ratings; 2) Aesthetic distribution refers to the distribution histogram of aesthetic scores of images; 3) Aesthetic attribute refers to the evaluation of good lighting, color harmony, shallow depth of field, balancing element, motion blur and other aspects of images; 4) Aesthetic description refers to linguistic aesthetics comments of images, as shown in Table 3.…”

“…Viswanatha et al [87] constructed a novel multi-task deep CNN with a merge-layer, which collects pooled features of the convolution maps to jointly learned eight aesthetic attributes along with the overall aesthetic score simultaneously. To understand the internal representation of these attributes in the learned model, they also develop the visual- .…”

“…Left: Visualizations of pixel-wise contributions to the classification of highly positive emotional (yellow) and negative emotional paintings (blue) [40]. Right: The predicted aesthetic score for a given attribute could be mapped back to the rectified convolution layers to generate the attribute activation maps [87] tion heat map, which is then used to highlight the important pixels for visual interpretation of the decision. The Activation Atlas [151] visualizes feature and attribution by using addition or interpolations between two neurons to demonstrate the semantic arithmetic properties of the activation space and how neurons jointly represent images.…”

“…The Activation Atlas [151] visualizes feature and attribution by using addition or interpolations between two neurons to demonstrate the semantic arithmetic properties of the activation space and how neurons jointly represent images. For explainable in deep aesthetic models, the predicted aesthetic score for a given attribute could be mapped back to the rectified convolution layers to generate the attribute activation maps, which highlight the attribute-specific discriminative regions [87], or visualizing how the aesthetic classifier "sees" paintings while judging the positive or negative emotions by using the back projection technique to display the relative pixel-wise contributions towards the specific task [40], [79], as shown in Figure 26.…”

Product Design Award Prediction Modeling: Design Visual Aesthetic Quality Assessment via DCNNs

“…Image aesthetics analysis can provide approach to predict the product design aesthetic level. In fact, image aesthetics computing has attracted many researchers' attention around the world [12][13][14][15][16]. Here, the relationship of visual aesthetic and image features were empirically estimated via computational algorithms in this study.…”