Continuous Human Activity Recognition With Distributed Radar Sensor Networks and CNN–RNN Architectures

Radar-based human motion and activity recognition is currently a topic of great research interest, as the aging population increases and older individuals prefer an independent lifestyle. This technology has a wide range of applications, such as fall detection in assisted living, gesture recognition for human-machine interfaces, and many more. Numerous studies exist on various approaches for radar-based activity capture and classification. However, most of these employ rather artificial data, often obtained in laboratory environments, and typically collected under particular conditions. Specifically, most research so far has aimed at distinguishing a predefined set of single activities with a defined start, stop and duration. This paper aims at drawing the attention to a so far less researched issue, one that will be of vital importance for future real-world application of radar-based human activity recognition: continuous activity recognition, i.e. recognizing specific activities in a stream of several sequential activities with unknown duration and arbitrary transitions between different classes of activities. A review on the current state of the art in this relatively new topic is given, followed by a discussion on future research directions.INDEX TERMS Radar applications, radar signal processing, continuous human activity recognition, activities of daily living.This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

show abstract

“…Another approach for classification of the TU Delft data set was published by Zhu et al [51] in 2022, with five main steps:…”

Section: The History Of Continuous Human Activity Recognitionmentioning

confidence: 99%

A Survey on Radar-Based Continuous Human Activity Recognition

Ullmann¹,

Guendel

Kruse

et al. 2023

IEEE J. Microw.

View full text Add to dashboard Cite

show abstract

“…Reference [23] proposes a multimodal sensor fusion framework based on a multilayer Bi-LSTM network, which fuses the data collected by wearable sensors and the data collected by FMCW radar and input it to the Bi-LSTM network at the same time. In [24], Zhu et al propose a hybrid classifier which combines both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) for spatialtemporal pattern extraction. Guendel et al [25] propose a coordinated network using five distributed pulsed ultrawideband (UWB) radars for continuous activities of daily living recognition in an arbitrary movement direction.…”

Section: Introductionmentioning

confidence: 99%

Continuous Human Motion Recognition Based on FMCW Radar and Transformer

Jiang

Che

et al. 2023

Journal of Sensors

View full text Add to dashboard Cite

Radar-based human motion recognition has received extensive attention in recent years. Most current recognition methods generate a heat map of features through simple signal processing and then feed into a classification-based neural network for recognition. Such an approach can only identify a single action. When a set of data contains information about multiple movements, it can also only be recognized as a single movement. Another point that cannot be overlooked is that continuous action recognition methods are able to recognize continuously changing actions but ignore the issue of whether continuous actions are legitimate or not (continuous actions obtained by stitching together multiple current actions do not conform to real time). In this paper, we propose a continuous action recognition method based on micro-Doppler features and transformer, which translates the micro-Doppler features of continuous actions into machine translation tasks and uses the idea of natural language processing (NLP) to identify continuous action. In order to judge whether the continuous action is legal or not, we also design the action state transition diagram as a constraint condition to strictly control the forward and backward actions. The experimental results show that the method proposed in this paper achieves good recognition accuracy for the recognition of a single action and can also effectively segment and recognize continuous actions.

show abstract

“…Hybrid networks, such as CNN-LSTM [ 14 , 15 , 16 , 17 , 18 , 19 ], can achieve enhanced performance compared to individual networks as they combine the expertise of the constituent networks. The hybrid structure can fully exploit the space–time characteristics of input data and improve the accuracy of recognition.…”

Section: Introductionmentioning

confidence: 99%

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

Huan

Zhang

et al. 2023

Sensors

View full text Add to dashboard Cite

Radar-based human activity recognition (HAR) provides a non-contact method for many scenarios, such as human–computer interaction, smart security, and advanced surveillance with privacy protection. Feeding radar-preprocessed micro-Doppler signals into a deep learning (DL) network is a promising approach for HAR. Conventional DL algorithms can achieve high performance in terms of accuracy, but the complex network structure causes difficulty for their real-time embedded application. In this study, an efficient network with an attention mechanism is proposed. This network decouples the Doppler and temporal features of radar preprocessed signals according to the feature representation of human activity in the time–frequency domain. The Doppler feature representation is obtained in sequence using the one-dimensional convolutional neural network (1D CNN) following the sliding window. Then, HAR is realized by inputting the Doppler features into the attention-mechanism-based long short-term memory (LSTM) as a time sequence. Moreover, the activity features are effectively enhanced using the averaged cancellation method, which improves the clutter suppression effect under the micro-motion conditions. Compared with the traditional moving target indicator (MTI), the recognition accuracy is improved by about 3.7%. Experiments based on two human activity datasets confirm the superiority of our method compared to traditional methods in terms of expressiveness and computational efficiency. Specifically, our method achieves an accuracy close to 96.9% on both datasets and has a more lightweight network structure compared to algorithms with similar recognition accuracy. The method proposed in this article has great potential for real-time embedded applications of HAR.

show abstract

Continuous Human Activity Recognition With Distributed Radar Sensor Networks and CNN–RNN Architectures

Cited by 43 publications

References 69 publications

A Survey on Radar-Based Continuous Human Activity Recognition

A Survey on Radar-Based Continuous Human Activity Recognition

Continuous Human Motion Recognition Based on FMCW Radar and Transformer

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

Contact Info

Product

Resources

About