Multi-View CNN-LSTM Architecture for Radar-Based Human Activity Recognition

Khalid, Habib-Ur-Rehman; Gorji, A. A.; Bourdoux, André; Pollin, Sofie; Sahli, Hichem

doi:10.1109/access.2022.3150838

Cited by 32 publications

(13 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1D-CNN applied in Ref. 36 gained an accuracy 94.60% while 37 applied a multi-view CNN-LSTM, resulting in an accuracy of 92.00%. Although these recent works have a great performance in radar-based human activity recognition, they still do not surpass the accuracy of our method through combining LSTM and PCA (99.1%).…”

Section: Discussionmentioning

confidence: 99%

Evaluation of deep learning models in contactless human motion detection system for next generation healthcare

Song

Taylor

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Recent decades have witnessed the growing importance of human motion detection systems based on artificial intelligence (AI). The growing interest in human motion detection systems is the advantages of automation in the monitoring of patients remotely and giving warnings to doctors promptly. Currently, wearable devices are frequently used for human motion detection systems. However, such devices have several limitations, such as the elderly not wearing devices due to lack of comfort or forgetfulness and/or battery limitations. To overcome the problems of wearable devices, we propose an AI-driven human motion detection system (deep learning-based system) using channel state information (CSI) extracted from Radio Frequency (RF) signals. The main contribution of this paper is to improve the performance of the deep learning models through techniques, including structure modification and dimension reduction of the original data. In this work, We firstly collected the CSI data with the center frequency 5.32 GHz and implemented the structure of the basic deep learning network in our previous work. After that, we changed the basic deep learning network by increasing the depth, increasing the width, adapting some advanced network structures, and reducing dimensions. After finishing those modifications, we observed the results and analyzed how to further improve the deep learning performance of this contactless AI-enabled human motion detection system. It can be found that reducing the dimension of the original data can work better than modifying the structure of the deep learning model.

show abstract

Section: Discussionmentioning

confidence: 99%

Evaluation of deep learning models in contactless human motion detection system for next generation healthcare

Song

Taylor

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Khalid et al. [ 19 ] proposed a multi-view CNN-LSTM architecture for the radar-based HMR, while the multi-view radar features includes raw features, energy dispersion based features, temporal difference based features, and auxiliary features.…”

Section: Related Workmentioning

confidence: 99%

Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Chen

Cheng

et al. 2022

Sensors

View full text Add to dashboard Cite

Doppler-radar-based continuous human motion recognition recently has attracted extensive attention, which is a favorable choice for privacy and personal security. Existing results of continuous human motion recognition (CHMR) using mmWave FMCW Radar are not considered the continuous human motion with the high similarity problem. In this paper, we proposed a new CHMR algorithm with the consideration of the high similarity (HS) problem, called as CHMR-HS, by using the modified Transformer-based learning model. As far as we know, this is the first result in the literature to investigate the continuous HMR with the high similarity. To obtain the clear FMCW radar images, the background and target signals of the detected human are separated through the background denoising and the target extraction algorithms. To investigate the effects of the spectral-temporal multi-features with different dimensions, Doppler, range, and angle signatures are extracted as the 2D features and range-Doppler-time and range-angle-time signatures are extracted as the 3D features. The 2D/3D features are trained into the adjusted Transformer-encoder model to distinguish the difference of the high-similarity human motions. The conventional Transformer-decoder model is also re-designed to be Transformer-sequential-decoder model such that Transformer-sequential-decoder model can successfully recognize the continuous human motions with the high similarity. The experimental results show that the accuracy of our proposed CHMR-HS scheme are 95.2% and 94.5% if using 3D and 2D features, the simulation results also illustrates that our CHMR-HS scheme has advantages over existing CHMR schemes.

show abstract

“…Hybrid networks, such as CNN-LSTM [ 14 , 15 , 16 , 17 , 18 , 19 ], can achieve enhanced performance compared to individual networks as they combine the expertise of the constituent networks. The hybrid structure can fully exploit the space–time characteristics of input data and improve the accuracy of recognition.…”

Section: Introductionmentioning

confidence: 99%

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

Huan

Zhang

et al. 2023

Sensors

View full text Add to dashboard Cite

Radar-based human activity recognition (HAR) provides a non-contact method for many scenarios, such as human–computer interaction, smart security, and advanced surveillance with privacy protection. Feeding radar-preprocessed micro-Doppler signals into a deep learning (DL) network is a promising approach for HAR. Conventional DL algorithms can achieve high performance in terms of accuracy, but the complex network structure causes difficulty for their real-time embedded application. In this study, an efficient network with an attention mechanism is proposed. This network decouples the Doppler and temporal features of radar preprocessed signals according to the feature representation of human activity in the time–frequency domain. The Doppler feature representation is obtained in sequence using the one-dimensional convolutional neural network (1D CNN) following the sliding window. Then, HAR is realized by inputting the Doppler features into the attention-mechanism-based long short-term memory (LSTM) as a time sequence. Moreover, the activity features are effectively enhanced using the averaged cancellation method, which improves the clutter suppression effect under the micro-motion conditions. Compared with the traditional moving target indicator (MTI), the recognition accuracy is improved by about 3.7%. Experiments based on two human activity datasets confirm the superiority of our method compared to traditional methods in terms of expressiveness and computational efficiency. Specifically, our method achieves an accuracy close to 96.9% on both datasets and has a more lightweight network structure compared to algorithms with similar recognition accuracy. The method proposed in this article has great potential for real-time embedded applications of HAR.

show abstract

Multi-View CNN-LSTM Architecture for Radar-Based Human Activity Recognition

Cited by 32 publications

References 28 publications

Evaluation of deep learning models in contactless human motion detection system for next generation healthcare

Evaluation of deep learning models in contactless human motion detection system for next generation healthcare

Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

Contact Info

Product

Resources

About