Human Sentiment and Activity Recognition in Disaster Situations Using Social Media Images Based on Deep Learning

Abstract

A rapidly increasing growth of social networks and the propensity of users to communicate their physical activities, thoughts, expressions, and viewpoints in text, visual, and audio material have opened up new possibilities and opportunities in sentiment and activity analysis. Although sentiment and activity analysis of text streams has been extensively studied in the literature, it is relatively recent yet challenging to evaluate sentiment and physical activities together from visuals such as photographs and videos. This paper emphasizes human sentiment in a socially crucial field, namely social media disaster/catastrophe analysis, with associated physical activity analysis. We suggest multi-tagging sentiment and associated activity analyzer fused with a a deep human count tracker, a pragmatic technique for multiple object tracking, and count in occluded circumstances with a reduced number of identity switches in disaster-related videos and images. A crowd-sourcing study has been conducted to analyze and annotate human activity and sentiments towards natural disasters and related images in social networks. The crowdsourcing study outcome into a large-scale benchmark dataset with three annotations sets each resolves distinct tasks. The presented analysis and dataset will anchor a baseline for future research in the domain. We believe that the proposed system will contribute to more viable communities by benefiting different stakeholders, such as news broadcasters, emergency relief organizations, and the public in general.

Exploring behavioural patterns and their relationships with social annotation outcomes

“…Additionally, we found no significant differences between initiators and responders in terms of positive tone and negative tone. Previously, sentiment analysis on social text (e.g., information published on social media) tended to focus on specific content or events that tend to arouse people's strong emotions or opinions, such as vaccination, disasters, and online learning during COVID‐19 (Mujahid et al, 2021; Sadiq et al, 2020). To our best knowledge, few studies have compared the sentiment of original and reply text.…”

“…tend to arouse people's strong emotions or opinions, such as vaccination, disasters, and online learning during COVID-19 (Mujahid et al, 2021;Sadiq et al, 2020). To our best knowledge, few studies have compared the sentiment of original and reply text.…”

Exploring Exploring Machine Learning and Deep Learning Approaches for Disaster Prediction and Management: A Survey of Different Approaches

“…Along with that a novel enhanced disaster prediction, assessment, and response system using UAVs along with path planning have been proposed. The author of the paper [6] introduces a deep learning technique that might be enhanced to distinguish between false and actual disasters using a mixed dataset and to read pictures and express views. Using NLP, ML, and DL, this work may also be done with text datasets and images.…”

A multimodal data analysis approach for social media during natural disasters

“…After a natural disaster, rescue organizations needed to use extensive data information in the initial phase as a decision basis to make low-risk decisions quickly (Sadiq, 2020), still the natural disasters could cause signal interruption, so obtaining useful data information became an urgent problem. In the past, because of the lack of data information, experts made emergency decisions mainly rely on their own knowledge experience (Xu, 2020).…”