Activity recognition of FMCW radar human signatures using tower convolutional neural networks

Abstract

Human activity recognition has become an obligatory necessity in day to day life and possible solutions can be provided with the technological advancement of sensing field. Radar based sensing with its unbeatable unique features has been a promising solution for identifying and distinguishing human activities in recent years. The ascent of loss of life among elderly people in care homes during COVID-19 is mainly due to poor monitoring services, that was not able to track their daily life activities. This has even more emphasized the need for savvy activity monitoring and tracking system. In this work, we have used a dataset that has captured six daily life activities of people from different locations during different times under realistic environments, unlike an regular controlled data collection environment. We have proposed a novel tower based convolutional neural network architecture that has employed parallel input layers with individual color channel images sent as inputs to the model. We have concatenated all the unique signature features from each channel to have better and robust feature representation to the model. We have analyzed the proposed model with different color spaces like RGB, LAB, HSV as inputs and found that our chosen input type performs better with the proposed model with significant test accuracy results. We have also compared our proposed model with other existing state of art architectures for radar based human activity recognition.

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

“…When implementing HAR using radar sensors, the MFFNs can be divided into featurelevel fusion networks [28][29][30][31][32][33][34][35] and decision-level fusion networks [36][37][38]. Feature-level fusion methods extract features from multiple inputs and combine them to create an even more comprehensive and richer feature representation.…”

“…Feature-level fusion methods extract features from multiple inputs and combine them to create an even more comprehensive and richer feature representation. For example, simple stitching operations on features [29][30][31][32][33][34][35] and feature fusion summation operations [28] are the most common methods.…”

“…Helen et al proposed a tower CNN that inputs three channels of red-green-blue (RGB) TD maps of human activities separately, with each color channel image as a parallel input layer. They utilized a splicing operation after the convolutional module and then learned the fused features using a 7-layer dense neural network [35]. Ding et al proposed a novel MFFN model with a summation approach.…”

Machine Learning for Healthcare Radars: Recent Progresses in Human Vital Sign Measurement and Activity Recognition

“…Another way of providing multi-domain radar to a CNN features extractor is presented by Helen et al [142] where a tower CNN consisting of parallel input layer was used in their work.…”

Micro-doppler radar to evaluate risk for musculoskeletal injury: Protocol for a case-control study with gold standard comparison

“…Another approach to gait analysis using MDR utilized trunk movement to estimate biomechanical gait parameters instead of the conventional leg-movement-based method [14]. Changes in leg positions relative to the radar make the extraction of ankle-echoes challenging, especially in practical conditions that do not involve walking on a treadmill [15][16][17]. Saho and colleagues demonstrated that the novel trunk method enables the estimation of the swing and stance times and that the inclusive trunk and leg-based method improves the accuracy of gait estimations [14].…”