Learning Subjective Attributes of Images from Auxiliary Sources

Gelli, Francesco; Uricchio, Tiberio; He, Xiangnan; Bimbo, Alberto Del; Chua, Tat-Seng

doi:10.1145/3343031.3350574

Cited by 8 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, in [ 26 ], an attention-based network, namely Attention-based Modality-Gated Networks (AMGN), has been proposed to exploit the correlation between visual and textual information for sentiment analysis. There are also some recent efforts for learning subjective attributes of images, such as emotions, from auxiliary sources of information (i.e., users’ interactions on social media) [ 27 ]. More recently, several attempts have been made for multi-level and multi-scale representation to extract visual cues of sentiment analysis.…”

Section: Related Workmentioning

confidence: 99%

Visual Sentiment Analysis from Disaster Images in Social Media

Hassan

Ahmad

Hicks

et al. 2022

Sensors

View full text Add to dashboard Cite

The increasing popularity of social networks and users’ tendency towards sharing their feelings, expressions, and opinions in text, visual, and audio content have opened new opportunities and challenges in sentiment analysis. While sentiment analysis of text streams has been widely explored in the literature, sentiment analysis from images and videos is relatively new. This article focuses on visual sentiment analysis in a societally important domain, namely disaster analysis in social media. To this aim, we propose a deep visual sentiment analyzer for disaster-related images, covering different aspects of visual sentiment analysis starting from data collection, annotation, model selection, implementation, and evaluations. For data annotation and analyzing people’s sentiments towards natural disasters and associated images in social media, a crowd-sourcing study has been conducted with a large number of participants worldwide. The crowd-sourcing study resulted in a large-scale benchmark dataset with four different sets of annotations, each aiming at a separate task. The presented analysis and the associated dataset, which is made public, will provide a baseline/benchmark for future research in the domain. We believe the proposed system can contribute toward more livable communities by helping different stakeholders, such as news broadcasters, humanitarian organizations, as well as the general public.

show abstract

Section: Related Workmentioning

confidence: 99%

Visual Sentiment Analysis from Disaster Images in Social Media

Hassan

Ahmad

Hicks

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The purpose of PIAA is to evaluate images by simulating the visual aesthetics of individual users [22]. Since users' aesthetic preferences are affected by multiple factors such as age, education, and behavioral habits [35,36], the PIAA for a specific user is more complicated and difficult than the GIAA for generic users. Due to the limited labeled samples provided by individuals, PIAA is a small sample learning task [28].…”

Section: Personalized Image Aesthetics Assessmentmentioning

confidence: 99%

Personalized Image Aesthetics Assessment via Multi-Attribute Interactive Reasoning

et al. 2022

View full text Add to dashboard Cite

Due to the subjective nature of people’s aesthetic experiences with respect to images, personalized image aesthetics assessment (PIAA), which can simulate the aesthetic experiences of individual users to estimate images, has received extensive attention from researchers in the computational intelligence and computer vision communities. Existing PIAA models are usually built on prior knowledge that directly learns the generic aesthetic results of images from most people or the personalized aesthetic results of images from a large number of individuals. However, the learned prior knowledge ignores the mutual influence of the multiple attributes of images and users in their personalized aesthetic experiences. To this end, this paper proposes a personalized image aesthetics assessment method via multi-attribute interactive reasoning. Different from existing PIAA models, the multi-attribute interaction constructed from both images and users is used as more effective prior knowledge. First, we designed a generic aesthetics extraction module from the perspective of images to obtain the aesthetic score distribution and multiple objective attributes of images rated by most users. Then, we propose a multi-attribute interactive reasoning network from the perspective of users. By interacting multiple subjective attributes of users with multiple objective attributes of images, we fused the obtained multi-attribute interactive features and aesthetic score distribution to predict personalized aesthetic scores. Experimental results on multiple PIAA datasets demonstrated our method outperformed state-of-the-art PIAA methods.

show abstract

“…For example, in (9) , the concept of an attention-based network known as Attention-based Modality-Gated Networks is presented to utilize the association among textual and visual data to do sentiment analysis. In more recent times, some attempts have been made to learn objective properties of photographs, like emotions, through auxiliary sources of data (10) . The feature extraction techniques used in this method cannot extract complex features such as structural dependency features and correlational features due to the system's high error rate issue.…”

Section: Introductionmentioning

confidence: 99%

Context Aware Image Sentiment Classification using Deep Learning Techniques

Agarwal¹,

Gupta²

2022

IJST

View full text Add to dashboard Cite

Objectives: To propose context aware sentiment classification using deep learning techniques. Methods: We used EfficientNetB-7 deep learning framework for caption generation for the input image and to classify the sentiment of generated caption using machine learning techniques. First, we employ several real-time and synthetic image datasets, then apply pre-processing and normalization for data balancing. Then efficient module implementation for feature extraction and selection using convolutional and pooling layers were done. Despite this proceeding, it generates the caption for respective images. The various feature extraction and selection Natural Language Processing (NLP) techniques such as TF-IDF, lemmas, dependency and correlational features have been used and classify the sentiment label using attention model and greedy approach. Finally, generating the blue score for the entire testing dataset and show the effectiveness of the proposed system. Findings: Our model gives higher accuracy with different deep learning techniques which is demonstrated in result section. The proposed model archives 73.80% average accuracy for EMOTIC dataset. The module has evaluated with different features and deep learning classification algorithms proposed earlier. Novelty: This research is the collaboration of Deep learning and machine learning classification techniques. We first extract the visual features from the input image using deep learning and classify with machine learning with the collaboration of NLP processes. We also carried out various feature extraction techniques such as Ngram, dependency features, co-relational features and determined the sentiment of generated captions.

show abstract

Learning Subjective Attributes of Images from Auxiliary Sources

Cited by 8 publications

References 25 publications

Visual Sentiment Analysis from Disaster Images in Social Media

Visual Sentiment Analysis from Disaster Images in Social Media

Personalized Image Aesthetics Assessment via Multi-Attribute Interactive Reasoning

Context Aware Image Sentiment Classification using Deep Learning Techniques

Contact Info

Product

Resources

About