A Hybrid Approach Based on GAN and CNN-LSTM for Aerial Activity Recognition

Abstract

Unmanned aerial vehicles (UAVs), known as drones, have played a significant role in recent years in creating resilient smart cities. UAVs can be used for a wide range of applications, including emergency response, civil protection, search and rescue, and surveillance, thanks to their high mobility and reasonable price. Automatic recognition of human activity in aerial videos captured by drones is critical for various tasks for these applications. However, this is difficult due to many factors specific to aerial views, including camera motion, vibration, low resolution, background clutter, lighting conditions, and variations in view. Although deep learning approaches have demonstrated their effectiveness in a variety of challenging vision tasks, they require either a large number of labelled aerial videos for training or a dataset with balanced classes, both of which can be difficult to obtain. To address these challenges, a hybrid data augmentation method is proposed which combines data transformation with the Wasserstein Generative Adversarial Network (GAN)-based feature augmentation method. In particular, we apply the basic transformation methods to increase the amount of video in the database. A Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) model is used to learn the spatio-temporal dynamics of actions, then a GAN-based technique is applied to generate synthetic CNN-LSTM features conditioned on action classes which provide a high discriminative spatio-temporal features. We tested our model on the YouTube aerial database, demonstrating encouraging results that surpass those of previous state-of-the-art works, including an accuracy rate of 97.83%.

CNN-ViT Supported Weakly-Supervised Video Segment Level Anomaly Detection

“…A LSTM network is a deep learning architecture based on an artificial recurrent neural network (RNN). It was specifically designed to handle sequential data, including videos, when modeling the short-range and long-range relationships of sequence features [ 66 ]. It also resolves the gradient vanishing problem of the RNN.…”

“…It is usually used for time series predictions [ 67 ]. However, to apply an LSTM network for temporal feature extraction, the output of the 2D-CNN spatial feature extractor can be fed to the LSTM network as input [ 66 ]. This can be performed by utilizing the output of the last fully connected layer of the 2D-CNN as the input for the LSTM.…”

Smart-Data-Glove-Based Gesture Recognition for Amphibious Communication

“…Specifically, the experiments collected samples of 5, 10, 20, 30, 40, and 50 instances for each gesture category. To verify the superiority of our proposed -DSNbased gesture recognition transfer model, we also selected several state-ofthe-art transfer learning models for comparison, including generative adversarial network (GAN)- [40] and conditional generative adversarial networks (CGAN)-based [41] transfer learning models. The transfer process involved utilizing our proposed DSN-based gesture recognition transfer model and selected state-of-the-art transfer learning models, with incremental updates applied to enhance the model's performance.…”

Detection and Recognition of Voice Commands by a Distributed Acoustic Sensor Based on Phase-Sensitive OTDR in the Smart Home Concept

“…Smart homes, smart manufacturing, and even smart cities are increasingly becoming part of the lives of citizens in many countries worldwide [1][2][3]. Some use these technologies as luxury items, but for people with disabilities, these are a few ways to improve the quality of one's life to an acceptable level.…”