TinyRadarNN: Combining Spatial and Temporal Convolutional Neural Networks for Embedded Gesture Recognition With Short Range Radars

Scherer, Moritz; Magno, Michele; Erb, Jonas; Mayer, Philipp; Eggimann, Manuel; Benini, Luca

doi:10.1109/jiot.2021.3067382

Cited by 66 publications

(29 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For comparison, in Scherer et al (2020), the authors developed a very low power embedded processing system for real-time gesture recognition based on radar sensing, which achieves 86.6-92.4% accuracy with energy consumption per classification of 4.52 mJ on inputs from a constellation of highresolution 60 GHz FMCW radars. One of the two datasets they consider (11-gesture) includes fine gestures with fingers, while the other one (5-gesture) contained more coarse-grained gestures analogous to ours.…”

Section: Radar-based Hand Gesture Classification In µBrainmentioning

confidence: 99%

μBrain: An Event-Driven and Fully Synthesizable Architecture for Spiking Neural Networks

et al. 2021

View full text Add to dashboard Cite

The development of brain-inspired neuromorphic computing architectures as a paradigm for Artificial Intelligence (AI) at the edge is a candidate solution that can meet strict energy and cost reduction constraints in the Internet of Things (IoT) application areas. Toward this goal, we present μBrain: the first digital yet fully event-driven without clock architecture, with co-located memory and processing capability that exploits event-based processing to reduce an always-on system's overall energy consumption (μW dynamic operation). The chip area in a 40 nm Complementary Metal Oxide Semiconductor (CMOS) digital technology is 2.82 mm2 including pads (without pads 1.42 mm2). This small area footprint enables μBrain integration in re-trainable sensor ICs to perform various signal processing tasks, such as data preprocessing, dimensionality reduction, feature selection, and application-specific inference. We present an instantiation of the μBrain architecture in a 40 nm CMOS digital chip and demonstrate its efficiency in a radar-based gesture classification with a power consumption of 70 μW and energy consumption of 340 nJ per classification. As a digital architecture, μBrain is fully synthesizable and lends to a fast development-to-deployment cycle in Application-Specific Integrated Circuits (ASIC). To the best of our knowledge, μBrain is the first tiny-scale digital, spike-based, fully parallel, non-Von-Neumann architecture (without schedules, clocks, nor state machines). For these reasons, μBrain is ultra-low-power and offers software-to-hardware fidelity. μBrain enables always-on neuromorphic computing in IoT sensor nodes that require running on battery power for years.

show abstract

Section: Radar-based Hand Gesture Classification In µBrainmentioning

confidence: 99%

μBrain: An Event-Driven and Fully Synthesizable Architecture for Spiking Neural Networks

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Wireless sensing is a promising and interesting technique which has various IoT applications and many researches have been conducted to exploit different types of wireless signals including mmWave signals [37][38][39][40][41] and WiFi signals [30,31,42] to recognize human hand gestures. These approaches vary in terms of signal characteristics, data preprocessing and neural network structure.…”

Section: Related Workmentioning

confidence: 99%

“…Secondly, the high frequency band of mmWave signal enables the strong anti-interference ability.Finally, it is easy to be embedded in portable devices due to the small size of mmWave radar chip [38]. Hence, many research efforts have been made to exploit mmWave signals for gesture recognition [38][39][40][41]. For instance, deep-soli [38] achieves fine-grained gesture recognition with a compact mmWave radar and deep neural network.…”

Section: Introductionmentioning

confidence: 99%

Towards Domain-Independent and Real-Time Gesture Recognition Using mmWave Signal

Li¹,

Zhang²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Human gesture recognition using millimeter wave (mmWave) signals provides attractive applications including smart home and in-car interface. While existing works achieve promising performance under controlled settings, practical applications are still limited due to the need of intensive data collection, extra training efforts when adapting to new domains (i.e. environments, persons and locations) and poor performance for real-time recognition. In this paper, we propose DI-Gesture, a domain-independent and real-time mmWave gesture recognition system. Specifically, we first derive the signal variation corresponding to human gestures with spatial-temporal processing. To enhance the robustness of the system and reduce data collecting efforts, we design a data augmentation framework based on the correlation between signal patterns and gesture variations. Furthermore, we propose a dynamic window mechanism to perform gesture segmentation automatically and accurately, thus enable real-time recognition. Finally, we build a lightweight neural network to extract spatial-temporal information from the data for gesture classification. Extensive experimental results show DI-Gesture achieves an average accuracy of 97.92%, 99.18% and 98.76% for new users, environments and locations, respectively. In real-time scenario, the accuracy of DI-Gesutre reaches over 97% with average inference time of 2.87ms, which demonstrates the superior robustness and effectiveness of our system.

show abstract

“…There are many types of 2D and 3D heatmaps, such as the Range-Doppler heatmap used in [9,24,33,34,39]. The frame dimension can be used as a separate dimension for the neural network to extract temporal information [11,24,33,34,39], or use together with other dimensions to be processed as part of the heatmap [5-7, 12, 36]. In our work we consider different heatmap based pipelines in previous works, together with other possible combinations, as our baselines.…”

Section: Introductionmentioning

confidence: 99%

CubeLearn: End-to-end Learning for Human Motion Recognition from Raw mmWave Radar Signals

Zhao¹,

Lu²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

mmWave FMCW radar has attracted huge amount of research interest for human-centered applications in recent years, such as human gesture/activity recognition. Most existing pipelines are built upon conventional Discrete Fourier Transform (DFT) pre-processing and deep neural network classifier hybrid methods, with a majority of previous works focusing on designing the downstream classifier to improve overall accuracy. In this work, we take a step back and look at the pre-processing module. To avoid the drawbacks of conventional DFT pre-processing, we propose a learnable preprocessing module, named CubeLearn, to directly extract features from raw radar signal and build an end-to-end deep neural network for mmWave FMCW radar motion recognition applications. Extensive experiments show that our CubeLearn module consistently improves the classification accuracies of different pipelines, especially benefiting those previously weaker models. We provide ablation studies on initialization methods and structure of the proposed module, as well as an evaluation of the running time on PC and edge devices. This work also serves as a comparison of different approaches towards data cube slicing. Through our task agnostic design, we propose a first step towards a generic end-to-end solution for radar recognition problems. CCS CONCEPTS• Computing methodologies → Machine learning.

show abstract

TinyRadarNN: Combining Spatial and Temporal Convolutional Neural Networks for Embedded Gesture Recognition With Short Range Radars

Cited by 66 publications

References 46 publications

μBrain: An Event-Driven and Fully Synthesizable Architecture for Spiking Neural Networks

μBrain: An Event-Driven and Fully Synthesizable Architecture for Spiking Neural Networks

Towards Domain-Independent and Real-Time Gesture Recognition Using mmWave Signal

CubeLearn: End-to-end Learning for Human Motion Recognition from Raw mmWave Radar Signals

Contact Info

Product

Resources

About