Detection of Emotions in Artworks Using a Convolutional Neural Network Trained on Non-Artistic Images: A Methodology to Reduce the Cross-Depiction Problem

Facial Expression Recognition (FER) has been widely explored in realistic settings; however, its application to artistic portraiture presents unique challenges due to the stylistic interpretations of artists and the complex interplay of emotions conveyed by both the artist and the subject. This study addresses these challenges through three key contributions. First, we introduce the PortraitEmotion3D (PE3D) dataset, designed explicitly for FER tasks in artistic portraits. This dataset provides a robust foundation for advancing emotion recognition in visual art. Second, we propose an innovative 3D emotion estimation method that leverages three-dimensional labeling to capture the nuanced emotional spectrum depicted in artistic works. This approach surpasses traditional two-dimensional methods by enabling a more comprehensive understanding of the subtle and layered emotions often in artistic representations. Third, we enhance the feature learning phase by integrating a self-attention module, significantly improving facial feature representation and emotion recognition accuracy in artistic portraits. This advancement addresses this domain’s stylistic variations and complexity, setting a new benchmark for FER in artistic works. Evaluation of the PE3D dataset demonstrates our method’s high accuracy and robustness compared to existing state-of-the-art FER techniques. The integration of our module yields an average accuracy improvement of over 1% in recent FER systems. Additionally, combining our method with ESR-9 achieves a comparable accuracy of 88.3% on the FER+ dataset, demonstrating its generalizability to other FER benchmarks. This research deepens our understanding of emotional expression in art and facilitates potential applications in diverse fields, including human–computer interaction, security, healthcare diagnostics, and the entertainment industry.

show abstract

Developing an AI-Based Digital Biophilic Art Curation to Enhance Mental Health in Intelligent Buildings

Xing,

Kar,

Bird

et al. 2024

Sustainability

View full text Add to dashboard Cite

Biophilic design is a well-recognised discipline aimed at enhancing health and well-being, however, most buildings lack adequate representation of nature or nature-inspired art. Notable barriers exist such as wealth, education, and physical ability restricting people’s accessibility to nature and associated artworks. An AI-based Biophilic arts curation and personalised recommendation system were developed in this study to improve accessibility to biophilic arts. Existing Biophilic research mainly focuses on building design principles, limited research exists to examine biophilic arts and associated emotional responses. In this paper, an interdisciplinary study addresses this gap by developing metrics for Biophilic art attributes and potential emotional responses, drawing on existing Biophilic architecture attributes and PANAS items. A public survey of 200 participants was developed in this study. The survey collected art viewers’ ratings of Biophilic attributes and associated emotional responses to establish statistical correlations between Biophilic attributes and emotional responses. The statistical analysis established a positive correlation between Biophilic attributes and positive emotions. The public survey results show significant positive emotional impacts (p-value <0.05) after exposure to Biophilic images, supporting further research and development of the Biophilic art curation system. This digital curation system employs Computer Vision algorithms (ResNet50) to automate Biophilic art categorisation and generate personalised recommendations. This study emphasises the importance of integrating nature into built environments. It proposes that artificial intelligence could significantly enhance the categorisation and recommendation of Biophilic art, advocating for expanding Biophilic art databases for emotionally responsive art display systems, benefiting mental health, and making art more accessible.

show abstract

Detection of Emotions in Artworks Using a Convolutional Neural Network Trained on Non-Artistic Images: A Methodology to Reduce the Cross-Depiction Problem

Cited by 3 publications

References 107 publications

Enhancing Facial Emotion Level Recognition: A CNN-Based Approach to Balancing Data

Enhancing Facial Emotion Level Recognition: A CNN-Based Approach to Balancing Data

PortraitEmotion3D: A Novel Dataset and 3D Emotion Estimation Method for Artistic Portraiture Analysis

Developing an AI-Based Digital Biophilic Art Curation to Enhance Mental Health in Intelligent Buildings

Contact Info

Product

Resources

About