Real-Time People Tracking and Identification From Sparse mm-Wave Radar Point-Clouds

Abstract

Mm-wave radars have recently gathered significant attention as a means to track human movement and identify subjects from their gait characteristics. A widely adopted method to perform the identification is the extraction of the micro-Doppler signature of the targets, which is computationally demanding in case of co-existing multiple targets within the monitored physical space. Such computational complexity is the main problem of state-of-the-art approaches, and makes them inapt for real-time use. In this work, we present an end-to-end, low-complexity but highly accurate method to track and identify multiple subjects in real-time using the sparse point-cloud sequences obtained from a low-cost mm-wave radar. Our proposed system features an extended object tracking Kalman filter, used to estimate the position, shape and extension of the subjects, which is integrated with a novel deep learning classifier, specifically tailored for effective feature extraction and fast inference on radar point-clouds. The proposed method is thoroughly evaluated on an edge-computing platform from NVIDIA (Jetson series), obtaining greatly reduced execution times (reduced complexity) against the best approaches from the literature. Specifically, it achieves accuracies as high as 91.62%, operating at 15 frames per seconds, in identifying three subjects that concurrently and freely move in an unseen indoor environment, among a group of eight.INDEX TERMS mm-wave radar, person identification, point-clouds, multi-target tracking, convolutional neural networks

Multiple Object Tracking in Deep Learning Approaches: A Survey

“…Second, they get similarity confidence from binary cross-entropy loss. Pegoraro and Rossi used cloud sequence by Mm-wave radars and used the extended Kalman filter [124], and the platform was evaluated in an edge-computing system. In Liu, tracking using deep learning-based detectors was performed using the Deep Associated Elastic Tracker (DAE-Tracker) [125].…”

“…We present MOT algorithms in six categories as shown in Figure 17. Ning et al [41] Zhu et al [32] Xiang et al [93] Milan et al [19] Azimi et al [93] Zhu et al [31] Liang et al [31] Yu et al [56] Chu et al [58] Leal-Taixe et al [76] Yoon et al [73] Azimi et al [93] Wang et al [132] Lee and Kim [117] Bewley et al [33] Weng and Kitani [55] Ray and Chakraborty [44] Pegoraro and Rossi [124] Hossain and Lee [115] Huang et al [129]…”

“…Ray and Chakraborty used only the Kalman filter in their tracking system [44]. Pegoraro and Rossi used the Kalman filter, classifier, and radar point cloud in their tracking system [124]. In Hossain and Lee, they combined CNN and the Kalman filter [115] for tracking.…”

Research on the General Algorithm and Benefit of Real-Time Positioning Advertising System—Based on the Use of 5G Base Station Data

“…For private enterprises, security issues are always the most business opportunities [22,23]. If we take an apartment complex as an example, several common safety issues are related to real-time footprint tracking.…”

mmFiT: Contactless Fitness Tracker Using mmWave Radar and Edge Computing Enabled Deep Learning

“…Most state of the art approaches require raw radar data for spectrogram generation. These methods are not suited for real-time operations on resource limited edge devices, as spectrogram generation is computationally expensive and raw data collection requires high data transfer speed [14]. Moreover, using only micro-doppler spectrograms may not be an effective approach for classification of symmetric activities since the spectrograms will be similar.…”