Rating Image Aesthetics Using a Crowd Sourcing Approach

Abstract

Abstract. Any system that is able to reliably measure the aesthetic appeal of photographs would be of considerable importance to the digital imaging industry. Researchers have built automated rating systems using machine learning techniques applied to features extracted from images. In this paper, we study the effectiveness of ACQUINE, a comprehensive and publicly available rating system, using data obtained from voters in a crowd sourced manner. We analyze the effect of voting using a simple binary like/dislike rating in comparison to a numerical 10 point scale. We also show that global measures of image quality, such as contrast or colorfulness, do not correlate well with human ratings. The role of composition in determining human rating of aesthetics is discussed.

Comparing humans to automation in rating photographic aesthetics

“…Figure 4 shows a histogram of the percentage of likes on the CSE dataset from the AMT workers in the study by Agrawal et al 6 The majority of the histogram is above 50%, peaking at 85%, which shows that workers tended to like more than dislike images.…”

“…Agrawal et al 6 checked if the percentage of likes provided by AMT workers in their study is in any way related to global statistics measuring contrast, colorfulness, and the rule of thirds in composition. Those statistics are calculated as follows.…”

“…Given these confounding factors, we critique and propose alternatives to current techniques based on our previous work. [4][5][6]…”

A Comprehensive Survey on Computational Aesthetic Evaluation of Visual Art Images: Metrics and Challenges

“…Many attempts have been made to contribute publicly available largescale datasets for more standardized evaluation of model performance. In the acquisition of subjective scores of image aesthetics, it can be realized through manually scoring experiments in the lab [38]- [40], online scoring on image sharing website [26], [41], and crowdsourcing evaluation [42], [43]. In the following, we introduce some most commonly used benchmark datasets for image aesthetic assessment in photographs and paintings, respectively.…”

“…Chen et al [89] proposed a double-column CNNs for learning aesthetic feature representation, which uses a weakly-supervised learning algorithm to project a set of textual attributes learned from image labels to highly responsive image regions. Such The scheme reported in [42] was able to maintain spatial relationships among objects but related background information and object attributes were not addressed. The scheme reported in [17] considers both objects and their interrelations, but have not been integrated with the holistic background modeling.…”