Convolutional Neural Networks as a Computational Model for the Underlying Processes of Aesthetics Perception

Denzler, Joachim; Rodner, Erik; Simon, Marcel

doi:10.1007/978-3-319-46604-0_60

Cited by 11 publications

(13 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Denzler et al ( 2016 ) proposed to use CNNs as model of perception for research in aesthetics. They trained the AlexNet model (Krizhevsky et al, 2012 ) on different datasets to experimentally evaluate how well pre-learned features of different layers are suited to distinguish art from non-art images using an SVM classifier.…”

Section: Computational Aesthetics: Algorithms and Applicationsmentioning

confidence: 99%

Computational and Experimental Approaches to Visual Aesthetics

Brachmann

Redies

2017

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Aesthetics has been the subject of long-standing debates by philosophers and psychologists alike. In psychology, it is generally agreed that aesthetic experience results from an interaction between perception, cognition, and emotion. By experimental means, this triad has been studied in the field of experimental aesthetics, which aims to gain a better understanding of how aesthetic experience relates to fundamental principles of human visual perception and brain processes. Recently, researchers in computer vision have also gained interest in the topic, giving rise to the field of computational aesthetics. With computing hardware and methodology developing at a high pace, the modeling of perceptually relevant aspect of aesthetic stimuli has a huge potential. In this review, we present an overview of recent developments in computational aesthetics and how they relate to experimental studies. In the first part, we cover topics such as the prediction of ratings, style and artist identification as well as computational methods in art history, such as the detection of influences among artists or forgeries. We also describe currently used computational algorithms, such as classifiers and deep neural networks. In the second part, we summarize results from the field of experimental aesthetics and cover several isolated image properties that are believed to have a effect on the aesthetic appeal of visual stimuli. Their relation to each other and to findings from computational aesthetics are discussed. Moreover, we compare the strategies in the two fields of research and suggest that both fields would greatly profit from a joined research effort. We hope to encourage researchers from both disciplines to work more closely together in order to understand visual aesthetics from an integrated point of view.

show abstract

Section: Computational Aesthetics: Algorithms and Applicationsmentioning

confidence: 99%

Computational and Experimental Approaches to Visual Aesthetics

Brachmann

Redies

2017

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…In recent years, computational aesthetics has progressed thanks to the use of generic features developed for other purposes in computer vision like object detection and classification or image retrieval. This development has reached a zenith with the development and widespread use of deep neural networks, in particular convolutional neural networks (CNN) [106][107][108]. Nevertheless, little attempts have been made to apply CNNs as the underlying model for aesthetic perception [107].…”

Section: Discussionmentioning

confidence: 99%

Aesthetic impact of solar energy systems

Sánchez-Pantoja

Vidal

Pastor

2018

Renewable and Sustainable Energy Reviews

View full text Add to dashboard Cite

The presence of solar energy systems has increased significantly in recent years both in rural areasin the form of solar farms-, and in urban areas as part of building installations. This transformation of the landscape, in spite of the good social acceptance of solar energy, causes an aesthetic impact whose interest has been growing in literature in recent years. This study aimed to review prior literature in order to establish the objective factors, aesthetic perception and methods that are most relevant when assessing the aesthetic impact. As a result of the lack of consensus, a new qualitative theoretical framework is proposed that can serve as a basis for future research in the field of the integration of solar energy and its aesthetic impact. The framework comprises three sub-impacts: land use, solar system energy and glare. The results are discussed for future research and innovation in building photovoltaic integration and for SES site location and its environmental impact assessments. Number of words excluding table/figure captions and references: 9686.

show abstract

“…They allow the creation of models that can analyze any picture and predict their aesthetic value, without the need for any annotated data about its contents; and without making use of hand-crafted features. Some examples of the use of CNNs for image aesthetics prediction and related topics can be found in [2,20,33,6,11,4,9,35,10,17]. Some of those papers make use of information about the contents of the pictures to improve the predictions of the models.…”

Section: Related Work 21 Computational Aesthetic Assessment In Photomentioning

confidence: 99%

Personalised Aesthetics with Residual Adapters

Rodríguez-Pardo

Bilen

2019

Pattern Recognition and Image Analysis

View full text Add to dashboard Cite

The use of computational methods to evaluate aesthetics in photography has gained interest in recent years due to the popularization of convolutional neural networks and the availability of new annotated datasets. Most studies in this area have focused on designing models that do not take into account individual preferences for the prediction of the aesthetic value of pictures. We propose a model based on residual learning that is capable of learning subjective, userspecific preferences over aesthetics in photography, while surpassing the stateof-the-art methods and keeping a limited number of user-specific parameters in the model. Our model can also be used for picture enhancement, and it is suitable for content-based or hybrid recommender systems in which the amount of computational resources is limited.The problem of taking into account subjective preferences on image aesthetics prediction is referred to as personalized image aesthetics [27]. Most recent approaches to image aesthetics evaluation have used different deep-learning models, which require a significant amount of annotated data for their training and evaluation. In real-world situations, it is unrealistic to assume that we will have thousands of annotated examples of rated images for any given user. This puts limits on the use of deep learning models for personalized image aesthetics prediction.In order to train a machine learning model capable of taking into account individual preferences over aesthetics in photography, an annotated dataset with the identities of the raters of each picture is needed. One example of this kind of dataset is the FLICKER-AES dataset, presented by Ren et al. [27], which contains over 40000 images rated by more than 200 different human raters. Their study provides, along with this dataset (and another, smaller, dataset), a residual-based learning model capable of taking into account user-specific preferences over aesthetics in photography.We build on their work, and propose an end-to-end convolutional neural network model capable of modelling user-specific preferences with different levels of abstraction, while keeping a reduced number of user-specific parameters within the model. Our method models user-specific preferences by using residual adapters, which were presented in [26,25] and have shown success in multi-domain learning. The main difference between our model and Ren et al.'s is that they model user-specific preferences by first training a generic aesthetics network, which predicts a mean aesthetic score, and computes a user-specific offset by training a Support Vector Regressor using the predicted content and some manually-defined attributes of the picture as its input; whereas our model embeds the user-specific parameters in different layers of the neural network, therefore allowing the model to find user-specific features with different levels of abstraction, and which do not necessarily depend on the contents and a fixed set of attributes of the pictures.Our main contributions are as follows: First, we propo...

show abstract

Convolutional Neural Networks as a Computational Model for the Underlying Processes of Aesthetics Perception

Cited by 11 publications

References 45 publications

Computational and Experimental Approaches to Visual Aesthetics

Computational and Experimental Approaches to Visual Aesthetics

Aesthetic impact of solar energy systems

Personalised Aesthetics with Residual Adapters

Contact Info

Product

Resources

About