Multiple Patients Behavior Detection in Real-time using mmWave Radar and Deep CNNs

Abstract

To address potential gaps noted in patient monitoring in the hospital, a novel patient behavior detection system using mmWave radar and deep convolution neural network (CNN), which supports the simultaneous recognition of multiple patients' behaviors in real-time, is proposed. In this study, we use an mmWave radar to track multiple patients and detect the scattering point cloud of each one. For each patient, the Doppler pattern of the point cloud over a time period is collected as the behavior signature. A three-layer CNN model is created to classify the behavior for each patient. The tracking and point clouds detection algorithm was also implemented on an mmWave radar hardware platform with an embedded graphics processing unit (GPU) board to collect Doppler pattern and run the CNN model. A training dataset of six types of behavior were collected, over a long duration, to train the model using Adam optimizer with an objective to minimize cross-entropy loss function. Lastly, the system was tested for real-time operation and obtained a very good inference accuracy when predicting each patient's behavior in a two-patient scenario.

mm-Pose: Real-Time Human Skeletal Posture Estimation Using mmWave Radars and CNNs

“…Studies have previously made use of micro-doppler signatures to determine human behavior using RF signals, however it did not provide spatial information of the subjects' locations [25], [26] as the signatures solely represented the temporal velocity profiles of the reflection points. Skeleton tracking using RF signals is a new and emerging area of research.…”

Millimeter Wave Sensing: A Review of Application Pipelines and Building Blocks

“…and Cartesian coordinates that fall outside certain boundaries [43], removing measured profile and spectrogram (part) averages from profiles and spectrograms [50], [81], [102], [138], deleting a profile belonging to an empty FOV from target measurement profiles [86], and statically removing the 0 Hz component from maximum Doppler frequency data across time [116]. Moving Target Indication (MTI) is not explained in [53], [58]. The papers suggest that it is a more general term used to indicate that denoising is performed.…”

“…Laplacian of Gaussian and Canny edge detection are used on a gray scale converted spectrogram containing magnitude values to discover Doppler frequency information [116]. Once range and AoA values retrieved from spectrograms or cubes are known, the values can be transformed from range and AoA information, through polar or spherical to two or three dimensional Cartesian coordinate conversion equations, to position information in Cartesian coordinates [43], [44], [46], [53], [55], [71], [93], [96], [103], [114], [121], [138]. The new data structure is referred to as either a point cloud frame or point scan.…”

“…Several Bayesian filters exist, such as the custom Bayesian [45], [185], particle [44], [136], α − β [128], Kalman [46], [53], [58], extended Kalman [55], [94], [96], [114], [115], [135], [186], fusion extended Kalman [95], [146], unscented Kalman [89], fusion adaptive Kalman [51], and adaptive Sage-Husa Kalman [52], [61] filter. The Kalman filter [219], under linear, quadratic, and Gaussian assumptions, can represent the state transition and observation functions x t = g(x t−1 , pn t ) and s t = h(x t , mn t ) as a set of linear equations.…”

3D Joints Estimation of the Human Body in Single-Frame Point Cloud

“…Some recent studies used micro-Doppler radar to determine human behavior with radio frequency signals, however it still cannot provide spatial information of the subject [29], [30]. Li et al [31] proposed a temporal Range-…”