Learning Multimodal Deep Representations for Crowd Anomaly Event Detection

Huang, Shaonian; Dong-jun, Huang; Zhou, Xiaojian

doi:10.1155/2018/6323942

Cited by 21 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experimental results showed the dominance of GET as compared to other models of image captioning. Experimental results on all listed MMID models are compiled and listed separately for each R-LSTM [39] RFNet [40] He [42] Feng [43] GET [44] Wang [45] FCN-LSTM [46] Bag-LSTM [47] Stack-VS [48] VSR [49] GLA [50] Up-Down [51] MAGAN [55] MGAN [56] R 0 5 10 15 20 25 30 35 40 RFNet [40] He [42] Feng [43] GET [44] Wang [45] FCN-LSTM [46] Bag-LSTM [47] Stack-VS [48] VSR [49] GLA [50] Up-Down [51] MAGAN [55] MGAN [56] S R-LSTM [39] RFNet [40] He [42] Feng [43] GET [44] Wang [45] FCN-LSTM [46] Bag-LSTM [47] Stack-VS [48] VSR [49] GLA [50] Up-Down R-LSTM [39] RFNet [40] He [42] Feng [43] GET [44] Wang [45] FCN-LSTM [46] Bag-LSTM…”

Section: Multimodal Image Description Resultsmentioning

confidence: 99%

“…CRNN [38] R-LSTM [39] RFNet [40] He [42] Feng [43] GET [44] Wang [45] FCN-LSTM [46] Bag-LSTM [47] Stack-VS [48] VSR [49] GLA [50] Up-Down [51] MAGAN [55] MGAN [56] M…”

Section: Multimodal Image Description Resultsmentioning

confidence: 99%

“…Images and text media are fed to a multi-instance deep network to classify events in this framework. S. Huang et al [44] proposed an unsupervised method for the detection of anomaly events from crowded scenes using DL architectures. In this model, visual, motion map and energy features are extracted from video frames.…”

Section: Multimodal Event Detection (Mmed)mentioning

confidence: 99%

See 2 more Smart Citations

A Review on Methods and Applications in Multimodal Deep Learning

Summaira¹,

Li²,

Shoib³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning (MMDL) is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This paper focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, and physiological signals. Detailed analysis of the baseline approaches and an in-depth study of recent advancements during the last five years (2017 to 2021) in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning methods is proposed, elaborating on different applications in more depth. Lastly, main issues are highlighted separately for each domain, along with their possible future research directions.CCS Concepts: • Computing methodologies → Machine learning; • Information systems → Multimedia and multimodal retrieval.

show abstract

Section: Multimodal Image Description Resultsmentioning

confidence: 99%

“…CRNN [38] R-LSTM [39] RFNet [40] He [42] Feng [43] GET [44] Wang [45] FCN-LSTM [46] Bag-LSTM [47] Stack-VS [48] VSR [49] GLA [50] Up-Down [51] MAGAN [55] MGAN [56] M…”

Section: Multimodal Image Description Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A Review on Methods and Applications in Multimodal Deep Learning

Summaira¹,

Li²,

Shoib³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Huang et al. [58] proposed also an ensemble of features, extracted using Convolutional Restricted Boltzmann Machines. Three different models were trained to extract information from visual patches (regions of original frames), energy patches (feature maps extracted by applying Gaussian filters to input patches), and motion patches (calculated using Optical Flow).…”

Section: Deep Learning For Crowd Anomaly Detection: Approaches and Numentioning

confidence: 99%

Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Sánchez

Hupont²,

Tabik

et al. 2020

Information Fusion

View full text Add to dashboard Cite

Crowd behaviour analysis is an emerging research area. Due to its novelty, a proper taxonomy to organise its different sub-tasks is still missing. This paper proposes a taxonomic organisation of existing works following a pipeline, where sub-problems in last stages benefit from the results in previous ones. Models that employ Deep Learning to solve crowd anomaly detection, one of the proposed stages, are reviewed in depth, and the few works that address emotional aspects of crowds are outlined. The importance of bringing emotional aspects into the study of crowd behaviour is remarked, together with the necessity of producing real-world, challenging datasets in order to improve the current solutions. Opportunities for fusing these models into already functioning video analytics systems are proposed.

show abstract

“…The experimental results show its competitive performance for anomaly event detection in video surveillance and there is a lot of scope to improve the detection accuracy using optimization. Shaonian Huang in 2018 [3] of traditional patterns. Then a multimodal fusion theme is employed to find out the deep representation of crowd patterns.…”

Section: Background Surveymentioning

confidence: 99%

An Optimized CNN based Real World Anomaly Detection in Surveillance Videos

2019

IJITEE

View full text Add to dashboard Cite

Anomaly detection in automated surveillance video is an extremely monotonous process for monitoring for crowded scenes and surveillance videos are capable to incarcerate a mixture of sensible anomalies. An appropriate machine learning technique can help to train the Anomaly Detection System (ADS) in identifying anomalous activities during surveillance. To this end, we present an anomaly detection system that can be used as a tool for anomaly detection in surveillance videos using the concept of artificial intelligence. The main intention of the proposed anomaly detection system is to improve the detection time and accuracy by using the concept of Convolutional Neural Network (CNN) as artificial intelligence technique. In this paper we present a CNN based Anomaly Detection System (CNN-ADS), which is the combination of multiple layer of hidden unit with the optimized MSER feature by using Genetic Algorithm (GA). Here CNN is used for classifying the activity into normal and abnormal from the surveillance videos based on the fitness function of GA which is used for the selection of optimal MSER feature sets. Further, Self adaptive genetic algorithm (SAGA) is adopted to efficiently solve optimization problems in the continuous search domain to select the best possible feature to segregate the pattern of normal and abnormal activities. The main contribution of this research is validation of proposed system for the large scale data and we introduce a new large-scale dataset of 128 hours of videos. Dataset consists of 1900 long and untrimmed real-world surveillance videos, with 13 sensible anomalies such as road accident, burglary, fighting, robbery, etc. as well as normal activities. The experimental results of the planned system show that our CNN-ADS for anomaly detection achieve essential improvement on anomaly detection presentation as compared to the state-of-the-art approaches. The dataset is available at: https://webpages.uncc.edu/cchen62/dataset.html. In this paper, to validate the proposed ADS we provide the comparison of existing results of several recent deep learning baselines on anomalous activity detection. The real-time ADS in surveillance video sequences using SAGA based CNN with MSER feature extraction technique is implemented using Image Processing Toolbox within Matlab Software.

show abstract

Learning Multimodal Deep Representations for Crowd Anomaly Event Detection

Cited by 21 publications

References 31 publications

A Review on Methods and Applications in Multimodal Deep Learning

A Review on Methods and Applications in Multimodal Deep Learning

Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

An Optimized CNN based Real World Anomaly Detection in Surveillance Videos

Contact Info

Product

Resources

About