Image sentiment analysis based on distillation and sentiment region localization network

Abstract

Accurately identifying the emotions in images is crucial for sentiment content analysis. To detect local sentiment regions and acquire discriminative sentiment features, we propose a novel model named Distillation-guided and Contrastive-enhanced Sentiment Region Localization Network (DC-SRLN) to effectively complete image sentiment analysis. Two smart but heterogeneous SRLNs are designed first to pursue local sentiment regions. Then an innovative contrastive learning mode is implemented between global and local features to further enhance the discriminative ability of the sentiment features. Third, the enhanced global and local sentiment features are seamlessly integrated to guide each SRLN accurately capture local sentiment regions. Finally, an adaptive feature fusion module is created to fuse the heterogeneous features from the two SRLNs and generate a new multi-view multi-granularity sentiment semantics with more discriminative ability for image sentiment analysis. Extensive experimental results on three prevailing datasets, namely Twitter I, FI, and ArtPhoto, exhibit that DC-SRLN achieves satisfactory accuracies of 93.2%, 80.6%, and 78.7%, respectively, outperforming recent state-of-the-art baselines. Moreover, DC-SRLN needs less training time, demonstrating its high practicality. The code of DC-SRLN is freely available at https://github.com/Riley6868/DC-SRLN.