Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar

Abstract

Doppler-radar-based continuous human motion recognition recently has attracted extensive attention, which is a favorable choice for privacy and personal security. Existing results of continuous human motion recognition (CHMR) using mmWave FMCW Radar are not considered the continuous human motion with the high similarity problem. In this paper, we proposed a new CHMR algorithm with the consideration of the high similarity (HS) problem, called as CHMR-HS, by using the modified Transformer-based learning model. As far as we know, this is the first result in the literature to investigate the continuous HMR with the high similarity. To obtain the clear FMCW radar images, the background and target signals of the detected human are separated through the background denoising and the target extraction algorithms. To investigate the effects of the spectral-temporal multi-features with different dimensions, Doppler, range, and angle signatures are extracted as the 2D features and range-Doppler-time and range-angle-time signatures are extracted as the 3D features. The 2D/3D features are trained into the adjusted Transformer-encoder model to distinguish the difference of the high-similarity human motions. The conventional Transformer-decoder model is also re-designed to be Transformer-sequential-decoder model such that Transformer-sequential-decoder model can successfully recognize the continuous human motions with the high similarity. The experimental results show that the accuracy of our proposed CHMR-HS scheme are 95.2% and 94.5% if using 3D and 2D features, the simulation results also illustrates that our CHMR-HS scheme has advantages over existing CHMR schemes.

TRANS-CNN-Based Gesture Recognition for mmWave Radar

“…Single-head attention has some limitations in extracting gesture features, and it is difficult to fully capture the gesture features in the sample data [ 37 ]. In contrast, in the multi-head attention mechanism, the Q , K , and V of each sub-network of the attention head are independent of each other, and different feature expressions can be learned in different subspaces.…”

Multi-person micro-Doppler signal decomposition based on millimeter wave radar

“…There are some researches on functional analysis of radar data, such as separating mD features of human trunk and limbs based on short-time fractional Fourier transform (STFrFT) and morphological component analysis (MCA) 3 . There are also some studies that fuse radar information of different dimensions, Chen 4 proposed a continuous human motion recognition (CHMR) algorithm. In order to distinguish highly similar human behaviors, they combined 2D features with 3D features, such as distance, angle, distance-Doppler time and distance-angle-time.…”

Vision Transformers (ViT) for Blanket-Penetrating Sleep Posture Recognition Using a Triple Ultra-Wideband (UWB) Radar System

“…A branch of deep learning models, vision transformers (ViTs), has emerged recently [33][34][35]. The origin of ViT, "Transformer", was designed for natural language processing (NLP) and was later applied to visual computing tasks, such as object detection [36] and segmentation [37], and human motion recognition [38], such as pose estimation [39,40], and gait recognition [41,42]. The "Transformer" built upon the sequence-to-sequence encoder-decoder architecture and substituted the recurrent layers with attention mechanism, enabling the long-term memory of every token (word) [43].…”