Avoiding the Inherent Limitations in Datasets Used for Measuring Aesthetics When Using a Machine Learning Approach

Abstract

An important topic in evolutionary art is the development of systems that can mimic the aesthetics decisions made by human begins, e.g., fitness evaluations made by humans using interactive evolution in generative art. This paper focuses on the analysis of several datasets used for aesthetic prediction based on ratings from photography websites and psychological experiments. Since these datasets present problems, we proposed a new dataset that is a subset of DPChallenge.com. Subsequently, three different evaluation methods were considered, one derived from the ratings available at DPChallenge.com and two obtained under experimental conditions related to the aesthetics and quality of images. We observed different criteria in the DPChallenge.com ratings, which had more to do with the photographic quality than with the aesthetic value. Finally, we explored learning systems other than state-of-the-art ones, in order to predict these three values. The obtained results were similar to those using state-of-the-art procedures.

Dataset for the Aesthetic Value Automatic Prediction

“…The correlation between the aesthetic and the quality criteria in the USA evaluations is 0.89, the same correlation that exists between criteria in the experiment carried out by Datta et al [3] With this new dataset, different models based on Machine Learning have been trained using different metrics for automatic prediction of aesthetic and quality value. The highest correlation obtained with these models is 0.58 using SVM [11].…”

Digital Image Quality Prediction System

“…The proposed dataset was built following the steps outlined in previous works [8]: obtaining the images on the web portal, filtering those images, organizing them according to their evaluation on the portal and selecting sets with an equal number of images. Subsequently, the quality of the images was evaluated by a group of humans through the Amazon Mechanical Turk platform.…”

Visual complexity modelling based on image features fusion of multiple kernels

“…Machine learning is widely used in several completely different fields; detection of regions of interests, image editing, texture analysis, visual aesthetic and quality assessment (Datta et al, 2006; Marchesotti et al, 2011; Romero et al, 2012; Fernandez-Lozano et al, 2015; Mata et al, 2018) and more recently in Carballal et al (2019b) or for microbiome analysis (Liu et al, 2017; Roguet et al, 2018), authentication of tequilas (Pérez-Caballero et al, 2017; Andrade et al, 2017), pathogenic point mutations (Rogers et al, 2018) or forensic identification (Gómez et al, 2018). Finally, with regard to the extraction of characteristics from images, some recently published works have been revised (Xu, Wang & Wang, 2018; Ali et al, 2016b; Wang et al, 2018; Sun et al, 2018; Zafar et al, 2018b).…”