Multi-Task Convolutional Neural Network for Image Aesthetic Assessment

The allure of aesthetic appeal in images captivates our senses, yet the underlying intricacies of aesthetic preferences remain elusive. In this study, we pioneer a novel perspective by utilizing several different machine learning (ML) models that focus on aesthetic attributes known to influence preferences. Our models process these attributes as inputs to predict the aesthetic scores of images. Moreover, to delve deeper and obtain interpretable explanations regarding the factors driving aesthetic preferences, we utilize the popular Explainable AI (XAI) technique known as SHapley Additive exPlanations (SHAP). Our methodology compares the performance of various ML models, including Random Forest, XGBoost, Support Vector Regression, and Multilayer Perceptron, in accurately predicting aesthetic scores, and consistently observing results in conjunction with SHAP. We conduct experiments on three image aesthetic benchmarks, namely Aesthetics with Attributes Database (AADB), Explainable Visual Aesthetics (EVA), and Personalized image Aesthetics database with Rich Attributes (PARA), providing insights into the roles of attributes and their interactions. Finally, our study presents ML models for aesthetics research, alongside the introduction of XAI. Our aim is to shed light on the complex nature of aesthetic preferences in images through ML and to provide a deeper understanding of the attributes that influence aesthetic judgements.

show abstract

Beyond pixels: text-guided deep insights into graphic design image aesthetics

Shi,

Li,

Song

2024

J. Electron. Imag.

View full text Add to dashboard Cite

The rapid development of computer vision and deep learning has significantly advanced image aesthetic assessment, yet traditional methods, which primarily rely on low-level visual features such as color and texture, often struggle with the complexity of graphic design images. These images are characterized by diverse design elements, including color, typography, and layout, as well as various styles such as minimalism, retro, and modernism, presenting substantial challenges to conventional assessment techniques. To overcome these limitations, we propose an innovative multimodal learning approach that integrates image content with textual descriptions to comprehensively analyze the aesthetic qualities of graphic design images. The core innovation of our method lies in the utilization of two distinct textual description methodologies: holistic descriptions, which capture the main theme of the design, and detailed descriptions, which focus on specific aspects such as composition, color, detail, and atmosphere. This dual approach allows for a more nuanced and complete assessment of aesthetic value. To effectively merge these descriptions with visual content, we introduce a feature similarity blending mechanism that aligns and integrates features from both modalities, enhancing the representation of aesthetic attributes. In addition, we employ a score bagging technique to aggregate scores from multiple fused features, ensuring robustness and reliability in the assessments. Our method is implemented within a multi-task learning framework, enabling simultaneous prediction across multiple rating dimensions. Experimental results demonstrate that, compared with the state-of-the-art TAHF method, our approach achieves notable improvements in Spearman's rank correlation coefficient-by 1.7%, 3.4%, and 2.6% on the HDDI, BAID, and TAD66K datasets, respectively-along with consistent gains in Pearson's linear correlation coefficient and accuracy. Moreover, our method achieves these performance improvements with fewer parameters and lower computational complexity, highlighting its efficiency and effectiveness in graphic design image aesthetic assessment.

show abstract

Multi-Task Convolutional Neural Network for Image Aesthetic Assessment

Cited by 6 publications

References 40 publications

Untitled

Untitled

Unveiling the factors of aesthetic preferences with explainable AI

Beyond pixels: text-guided deep insights into graphic design image aesthetics

Contact Info

Product

Resources

About