Visual Sentiment Analysis by Combining Global and Local Information

Wu, Lifang; Qi, Mei; Jian, Meng; Zhang, Heng

doi:10.1007/s11063-019-10027-7

Cited by 49 publications

(32 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning is by all means contributing to making polarity detectors very reliable. A very interesting research direction is the development of models capable of locating the most important regions in an image, to exploit saliency to improve sentiment analysis [48,[54][55][56][57][58]. Up to now, multi-task learning has received little attention, although several applications exist that require solving multiple visual tasks.…”

Section: Concluding Remarks and Discussionmentioning

confidence: 99%

“…In [48], the Vgg_19 architecture processed an image including the conventional R, G, and B channels and a focal channel; the latter channel was set to model human attention. Likewise, the works in [54,55] studied the relevance of salience in sentiment detection. Attention mechanisms [56,57] were addressed in [58], where the authors combined VggNet architectures with a recurrent neural network (RNN).…”

Section: Polarity Detectionmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational Costs

et al. 2019

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) provide an effective tool to extract complex information from images. In the area of image polarity detection, CNNs are customarily utilized in combination with transfer learning techniques to tackle a major problem: the unavailability of large sets of labeled data. Thus, polarity predictors in general exploit a pre-trained CNN as the feature extractor that in turn feeds a classification unit. While the latter unit is trained from scratch, the pre-trained CNN is subject to fine-tuning. As a result, the specific CNN architecture employed as the feature extractor strongly affects the overall performance of the model. This paper analyses state-of-the-art literature on image polarity detection and identifies the most reliable CNN architectures. Moreover, the paper provides an experimental protocol that should allow assessing the role played by the baseline architecture in the polarity detection task. Performance is evaluated in terms of both generalization abilities and computational complexity. The latter attribute becomes critical as polarity predictors, in the era of social networks, might need to be updated within hours or even minutes. In this regard, the paper gives practical hints on the advantages and disadvantages of the examined architectures both in terms of generalization and computational cost.

show abstract

Section: Concluding Remarks and Discussionmentioning

confidence: 99%

Section: Polarity Detectionmentioning

confidence: 99%

A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational Costs

et al. 2019

View full text Add to dashboard Cite

show abstract

“…GM EI &LRM SI [ 27 ]: The work found through research that not all images in the dataset contain salient objects. Therefore, this work believes that visual sentiment analysis should not only focus on local features.…”

Section: Methodsmentioning

confidence: 99%

“…On the basis of NASNet, Yadav et al applied the residual attention module to learn the important areas related to emotion in the image [ 26 ]. Wu et al suggested to use the object detection module to determine if use a local module [ 27 ]. Yang et al [ 28 ] proposed a method to finding related regions using a ready-made method to get object proposals as local emotional information and used VGG to learn global information.…”

Section: Related Workmentioning

confidence: 99%

Multi-Level Context Pyramid Network for Visual Sentiment Analysis

Qing

et al. 2021

Sensors

View full text Add to dashboard Cite

Sharing our feelings through content with images and short videos is one main way of expression on social networks. Visual content can affect people’s emotions, which makes the task of analyzing the sentimental information of visual content more and more concerned. Most of the current methods focus on how to improve the local emotional representations to get better performance of sentiment analysis and ignore the problem of how to perceive objects of different scales and different emotional intensity in complex scenes. In this paper, based on the alterable scale and multi-level local regional emotional affinity analysis under the global perspective, we propose a multi-level context pyramid network (MCPNet) for visual sentiment analysis by combining local and global representations to improve the classification performance. Firstly, Resnet101 is employed as backbone to obtain multi-level emotional representation representing different degrees of semantic information and detailed information. Next, the multi-scale adaptive context modules (MACM) are proposed to learn the sentiment correlation degree of different regions for different scale in the image, and to extract the multi-scale context features for each level deep representation. Finally, different levels of context features are combined to obtain the multi-cue sentimental feature for image sentiment classification. Extensive experimental results on seven commonly used visual sentiment datasets illustrate that our method outperforms the state-of-the-art methods, especially the accuracy on the FI dataset exceeds 90%.

show abstract

“…The system classifies a givne image using the eight emotions scheme proposed by Mikels et al [31]. The authors of [64] combined the use of CNNs and saliency detection to develop a system that first performs a prediction of the sentiment of the whole image. Then, if salient regions are detected, performs the sentiment prediction on the sub‐images that depict such sub‐areas.…”

Section: State‐of‐the‐artmentioning

confidence: 99%