No abstract

In this work we present RAPID, a Joint Communication & Radar (JCR) system based on next generation IEEE 802.11ay WiFi networks in the 60 GHz band, which retrofits existing communication hardware to perform zerocost monitoring of human movement and activities in indoor spaces. Most of the available approaches for human sensing at millimeter-waves employ special-purpose radar devices, which are utilized to retrieve and analyze the small-scale Doppler effect (micro-Doppler) caused by human motion. This is key to achieve fine-grained sensing applications such as simultaneous activity recognition and person identification. We show that IEEE 802.11ay Access Points (APs) can be retrofitted to perform radar-like extraction of micro-Doppler effects of multiple human subjects. While radar systems entail the deployment of additional hardware, our framework performs activity recognition and person identification using IEEE 802.11ay wireless networks with no modifications to the transmitted packet structure specified by the standard. We leverage the in-packet beam training capability of IEEE 802.11ay to accurately localize and track multiple individuals using the estimated Channel Impulse Response (CIR), while the beam tracking mechanism embedded in data packets allows to extract the desired micro-Doppler (µD) signatures of the subjects. We implement our system on an IEEE 802.11aycompatible full-duplex FPGA platform with phased antenna arrays, which can estimate the CIR from the reflections of transmitted packets. Using two access points, we achieve reliable positioning and tracking of multiple subjects, and an accuracy of 93% and 90% for activity recognition and person identification, respectively.

Mm-wave radars have recently gathered significant attention as a means to track human movement and identify subjects from their gait characteristics. A widely adopted method to perform the identification is the extraction of the micro-Doppler signature of the targets, which is computationally demanding in case of co-existing multiple targets within the monitored physical space. Such computational complexity is the main problem of state-of-the-art approaches, and makes them inapt for real-time use. In this work, we present an end-to-end, low-complexity but highly accurate method to track and identify multiple subjects in real-time using the sparse point-cloud sequences obtained from a low-cost mm-wave radar. Our proposed system features an extended object tracking Kalman filter, used to estimate the position, shape and extension of the subjects, which is integrated with a novel deep learning classifier, specifically tailored for effective feature extraction and fast inference on radar point-clouds. The proposed method is thoroughly evaluated on an edge-computing platform from NVIDIA (Jetson series), obtaining greatly reduced execution times (reduced complexity) against the best approaches from the literature. Specifically, it achieves accuracies as high as 91.62%, operating at 15 frames per seconds, in identifying three subjects that concurrently and freely move in an unseen indoor environment, among a group of eight.INDEX TERMS mm-wave radar, person identification, point-clouds, multi-target tracking, convolutional neural networks

Wideband millimeter-wave communication systems can be extended to provide radar-like sensing capabilities on top of data communication, in a cost-effective manner. However, the development of joint communication and sensing technology is hindered by practical challenges, such as occlusions to the line-of-sight path and clock asynchrony between devices. The latter introduces time-varying timing and frequency offsets that prevent the estimation of sensing parameters and, in turn, the use of standard signal processing solutions. Existing approaches cannot be applied to commonly used phased-array receivers, as they build on stringent assumptions about the multipath environment, and are computationally complex. We present JUMP, the first system enabling practical bistatic and asynchronous joint communication and sensing, while achieving accurate target tracking and micro-Doppler extraction in realistic conditions. Our system compensates for the timing offset by exploiting the channel correlation across subsequent packets. Further, it tracks multipath reflections and eliminates frequency offsets by observing the phase of a dynamically-selected static reference path. JUMP has been implemented on a 60 GHz experimental platform, performing extensive evaluations of human motion sensing, including non-line-of-sight scenarios. In our results, JUMP attains comparable tracking performance to a fullduplex monostatic system and similar micro-Doppler quality with respect to a phase-locked bistatic receiver.

We address the use of backscattered mm-wave radio signals to track humans as they move within indoor environments. The common approach in the literature leverages the extended Kalman filter (EKF) method, which however undergoes a severe performance degradation when the system evolution model is highly non-linear or presents long-term time dependencies among the system states. In this work, we propose an original model-free tracking procedure based on denoising autoencoders and sequence-to-sequence neural networks, showing its superior performance with respect to state-of-the-art methods. Our architecture can be trained in either a supervised or unsupervised manner, trading tracking accuracy for flexibility. The proposed system is tested on our own measurements, obtained with a 77 GHz radar on single and multiple subjects simultaneously moving in an indoor space. The results are compared against the ground truth trajectories from a motion tracking system, obtaining average tracking errors as low as 12 cm.