Joint regression and learning from pairwise rankings for personalized image aesthetic assessment

Abstract

Recent image aesthetic assessment methods have achieved remarkable progress due to the emergence of deep convolutional neural networks (CNNs). However, these methods focus primarily on predicting generally perceived preference of an image, making them usually have limited practicability, since each user may have completely different preferences for the same image. To address this problem, this paper presents a novel approach for predicting personalized image aesthetics that fit an individual user’s personal taste. We achieve this in a coarse to fine manner, by joint regression and learning from pairwise rankings. Specifically, we first collect a small subset of personal images from a user and invite him/her to rank the preference of some randomly sampled image pairs. We then search for the K-nearest neighbors of the personal images within a large-scale dataset labeled with average human aesthetic scores, and use these images as well as the associated scores to train a generic aesthetic assessment model by CNN-based regression. Next, we fine-tune the generic model to accommodate the personal preference by training over the rankings with a pairwise hinge loss. Experiments demonstrate that our method can effectively learn personalized image aesthetic preferences, clearly outperforming state-of-the-art methods. Moreover, we show that the learned personalized image aesthetic benefits a wide variety of applications.

A Visual Enhancement Network with Feature Fusion for Image Aesthetic Assessment

“…The number of those images account for 4.5% of all images. If Mean in the range [4,7), the score distribution of the corresponding image is largely Gaussian. Inspired by this, we divide AVA dataset into three parts depending on Mean and conduct the experiments, respectively.…”

“…Recently, deep learning has become a hot topic for IAA [3], which overcomes the limitation of hand-crafted feature extraction. Neural networks have shown good advantages for image analysis and processing [4]- [6]. Dai et al [7] introduced the existing research in the field of intelligent media.…”

AI‐driven user aesthetics preference prediction for UI layouts via deep convolutional neural networks

“…J. Zhou trained a fine‐tuned generic model by convolutional neural network (CNN)‐based regression for individual preference prediction. The experimental result shows that their method can predict personalised image aesthetic preference effectively [20]. W. Niu et al.…”

“…collected 0.3 million Flickr images for user preference modelling and proposed a novel hybrid algorithm, which integrated the image content analysis with user sparse information. The method outperformed another latent factor‐based method, with a mean Average percentile score of 45.77 [20, 26].…”

“…modelling and proposed a novel hybrid algorithm, which integrated the image content analysis with user sparse information. The method outperformed another latent factor-based method, with a mean Average percentile score of 45.77[20,26]. Both visual and text-based features were utilised to predict user 'likes' with a probabilistic approach[29].…”