CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks

Ullah, Waseem; Ullah, Amin; Haq, Ijaz Ul; Muhammad, Khan; Sajjad, Muhammad; Baik, Sung Wook

doi:10.1007/s11042-020-09406-3

Cited by 192 publications

(113 citation statements)

References 39 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…In this model, the frame-level features are extracted from the videos and then fed to a bidirectional LSTM to classify abnormal events at an automated teller machine. In our pioneering work, we used deep CNN features from a series of frames and passed them through a multilayer bidirectional LSTM to learn the spatiotemporal information of the input video and detect abnormal events [ 36 ]. Luo et al [ 18 ] suggested a convolutional LSTM with an autoencoder-based model for anomaly detection in videos.…”

Section: Related Workmentioning

confidence: 99%

“…This system is based on two flow feature networks: one uses CNN-based features while the other uses motion features separately. In our pioneering work, we used deep CNN features from the series of frames and passed them through a multilayer bidirectional LSTM to learn the spatiotemporal information of the input video and detect the abnormal events [ 36 ].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos

Ullah

Hussain

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Video anomaly recognition in smart cities is an important computer vision task that plays a vital role in smart surveillance and public safety but is challenging due to its diverse, complex, and infrequent occurrence in real-time surveillance environments. Various deep learning models use significant amounts of training data without generalization abilities and with huge time complexity. To overcome these problems, in the current work, we present an efficient light-weight convolutional neural network (CNN)-based anomaly recognition framework that is functional in a surveillance environment with reduced time complexity. We extract spatial CNN features from a series of video frames and feed them to the proposed residual attention-based long short-term memory (LSTM) network, which can precisely recognize anomalous activity in surveillance videos. The representative CNN features with the residual blocks concept in LSTM for sequence learning prove to be effective for anomaly detection and recognition, validating our model’s effective usage in smart cities video surveillance. Extensive experiments on the real-world benchmark UCF-Crime dataset validate the effectiveness of the proposed model within complex surveillance environments and demonstrate that our proposed model outperforms state-of-the-art models with a 1.77%, 0.76%, and 8.62% increase in accuracy on the UCF-Crime, UMN and Avenue datasets, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos

Ullah

Hussain

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Next, the spatio-temporal features collected from the output of the encoder are used for classification. Ullah et al [27] extracted spatio-temporal features from a series of frames by passing each one to a pre-trained convolutional neural network model. They then fed the extracted deep features to multilayer bi-directional long short-term memory model, which can classify ongoing anomalous/normal events in complex surveillance scenes.…”

Section: Related Workmentioning

confidence: 99%

IA-SSLM: Irregularity-Aware Semi-Supervised Deep Learning Model for Analyzing Unusual Events in Crowds

Aljaloud

Ullah

2021

IEEE Access

View full text Add to dashboard Cite

Analyzing unusual events is significantly important for video surveillance to ensure people safety. These events are characterized by irregular patterns that do not conform to the expected behavior in the surveillance scenes. We present a novel irregularity-aware semi-supervised deep learning model (IA-SSLM) for detection of unusual events. While most existing works depend on the availability of large amount of labeled data for training, our proposed method utilizes a semi-supervised deep model to automatically learn feature representations from limited number of labeled data samples. Our method extracts meaningful information from both labeled and unlabeled data during the training stage to improve the performance. For this purpose, we explore the concept of consistency regularization and entropy minimization to output confident predictions on unlabeled data. For experimental analysis, we consider various standard and diverse datasets. The results show that our IA-SSLM method outperforms several reference methods using different performance metrics.

show abstract

“…As shown in the table, our method achieved the best performance both in AUC and fps. To be specific, our method is about 2% and 6∼10% superior to BI-LSTM [36] and the methods in [15], [37], respectively. Our method is better by about 1-3% than the SG3I [10] in the AUC and is much faster in terms of the fps performance on NVIDIA Jetson Nano.…”

Section: Anomaly Detection In Edge-computing Environmentmentioning

confidence: 99%

Deep Edge Computing for Videos

Kim

Won³

2021

IEEE Access

View full text Add to dashboard Cite

This paper provides a modular architecture with deep neural networks as a solution for realtime video analytics in an edge-computing environment. The modular architecture consists of two networks of Front-CNN (Convolutional Neural Network) and Back-CNN, where we adopt Shallow 3D CNN (S3D) as the Front-CNN and a pre-trained 2D CNN as the Back-CNN. The S3D (i.e., the Front CNN) is in charge of condensing a sequence of video frames into a feature map with three channels. That is, the S3D takes a set of sequential frames in the video shot as input and yields a learned 3 channel feature map (3CFM) as output. Since the 3CFM is compatible with the three-channel RGB color image format, we can use the output of the S3D (i.e., the 3CFM) as the input to a pre-trained 2D CNN of the Back-CNN for the transfer-learning. This serial connection of Front-CNN and Back-CNN architecture is end-to-end trainable to learn both spatial and temporal information of videos. Experimental results on the public datasets of UCF-Crime and UR-Fall Detection show that the proposed S3D-2DCNN model outperforms the existing methods and achieves state-of-the-art performance. Moreover, since our Front-CNN and Back-CNN modules have a shallow S3D and a light-weighted 2D CNN, respectively, it is suitable for real-time video recognition in edge-computing environments. We have implemented our CNN model on NVIDIA Jetson Nano Developer as an edge-computing device to show its real-time execution.

show abstract

CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks

Cited by 192 publications

References 39 publications

An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos

An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos

IA-SSLM: Irregularity-Aware Semi-Supervised Deep Learning Model for Analyzing Unusual Events in Crowds

Deep Edge Computing for Videos

Contact Info

Product

Resources

About