Detecting Bias on Aesthetic Image Datasets

Abstract

In recent years, there have been attempts to discover the principles that determine the value of aesthetics in the domain of computing. Many and diverse studies have tried in some way to capture these principles through technical characteristics. To this end, helped by the ease of Internet data acquisition, datasets of images have been published which were obtained online at random from websites and photography competitions. To guarantee the validity of a system of aesthetic image classification, one must first guarantee its capacity for generalization. This paper studies how the indiscriminate selection of images can affect the generalization capacity obtained by a binary classifier.

Avoiding the Inherent Limitations in Datasets Used for Measuring Aesthetics When Using a Machine Learning Approach

“…Taking into account a number of problems found regarding the state-of-the-art datasets, a dataset was developed following a new methodology. This dataset consists of 1000 images from the DPChallenge portal, which were evaluated in 3 different ways: (1) evaluation from the DPChallenge portal with at least 100 scores per image; (2) an aesthetic evaluation conducted under controlled experimental conditions and a minimum of 10 votes per image; (3) a quality assessment made under the same conditions as (2). As far as the authors are aware, this is the first time a dataset is evaluated based on three different criteria by two different populations.…”

“…However, in experiments where the test was performed with a set from a source different from the training set, the correlation results decreased notably. A clear example in this regard can be seen in experiments conducted in previous research studies [1,2]: when training a subset of 6,000 images from DPChallenge.com carried out by Ke et Complexity 3 al. [5], the result of the correlation was 91.38%.…”

“…Previous research studies [1,2] have concluded that the degree of generalisation of some existing sample sets was not enough to take them as reference in the training of automated prediction and classification of images. Other functional limitations were identified in these datasets, which are also mentioned in this paper.…”

Dataset for the Aesthetic Value Automatic Prediction

“…Due the subjective nature of the aesthetic problem, the selection of the dataset with which the system is trained is especially relevant. After analyzed, in previous research [1,2], the generalization degree of some datasets, it has been concluded that it is not enough to take them as a reference in the training of automatic image classification and prediction systems. In order to providing a solution to the problems detected, this paper describes the creation of a new dataset from the DPChallenge.com portal, with greater statistical coherence.…”