Few-Shot User-Definable Radar-Based Hand Gesture Recognition at the Edge

Abstract

Technological advances and scalability are leading Human-Computer Interaction (HCI) to evolve towards intuitive forms, such as through gesture recognition. Among the various interaction strategies, radar-based recognition is emerging as a touchless, privacy-secure, and versatile solution in different environmental conditions. Classical radar-based gesture HCI solutions involve deep learning but require training on large and varied datasets to achieve robust prediction. Innovative self-learning algorithms can help tackling this problem by recognizing patterns and adapt from similar contexts. Yet, such approaches are often computationally expensive and hardly integrable into hardware-constrained solutions. In this paper, we present a gesture recognition algorithm which is easily adaptable to new users and contexts. We exploit an optimization-based meta-learning approach to enable gesture recognition in learning sequences. This method targets at learning the best possible initialization of the model parameters, simplifying training on new contexts when small amounts of data are available. The reduction in computational cost is achieved by processing the radar sensed data of gestures in the form of time maps, to minimize the input data size. This approach enables the adaptation of simple convolutional neural network (CNN) to new hand poses, thus easing the integration of the model into a hardware-constrained platform. Moreover, the use of a Variational Autoencoders (VAE) to reduce the gestures' dimensionality leads to a model size decrease of an order of magnitude and to half of the required adaptation time. The proposed framework, deployed on the Intel ® Neural Compute Stick 2 (NCS 2), leads to an average accuracy of around 84% for unseen gestures when only one example per class is utilized at training time. The accuracy increases up to 92.6% and 94.2% when three and five samples per class are used.

Analysis of User-Defined Radar-Based Hand Gestures Sensed Through Multiple Materials

“…In the user-dependent scenario, the best recognition rate with wood is 96.1% with the configuration T =4, N =33, and AP= (1,2,3,6,8,9), resulting in an execution time of 5.05 ms, thereby making it eligible for dynamic, real-time interaction. With the PVC, the best recognition rate is 83.7% with the configuration T =4, N =38, and AP= (4,5,7,10,11,12) for an execution time of 5.72 ms. Gestures performed in front of the glass are recognized with a rate of 67% with the configuration T =4, N =38, and AP= (1,2,3,4,5,6,7,8,9,10,11,12), leading to an execution time of 8.47 ms.…”

“…With the PVC, the best recognition rate is 83.7% with the configuration T =4, N =38, and AP= (4,5,7,10,11,12) for an execution time of 5.72 ms. Gestures performed in front of the glass are recognized with a rate of 67% with the configuration T =4, N =38, and AP= (1,2,3,4,5,6,7,8,9,10,11,12), leading to an execution time of 8.47 ms. Fig.…”

“…In view of all this work, we can apply a DL algorithm, but we are keen to put a template-based [50] recognizer [51] to the test because it allows us to customize the gestures by modifying templates only, with a reasonable number of templates [12], and without re-training [52], [53]. Furthermore, we are siding with the mobile radars that are widely available on the market by taking the Walabot DIY 2 radar [3], [5], [15], [26], [45], [54], [55].…”

“…• Number of sampling points (N ): represents the number of points per gesture template: N ={x ∈ N|4≤x≤40}. • Antenna pairs (AP): defines the set of antenna pairs from which the data were used: AP={ (1,2,3,6,8,9), (4,5,7,10,11,12), (1,2,3,4,5,6,7,8,9,10,11,12)}. These three sets demonstrated admissible results [26].…”

“…On one hand, radar-based gesture recognition systems have demonstrated their effectiveness [10], reliability, and robustness [11] despite being influenced by environmental conditions [12]. Most studies [12]- [18] have reported their perfor-mance in terms of high recognition accuracy rates, making dynamic [16], [17], real-time [19]- [21] interaction feasible. High-frequency, wide-bandwidth radars detect even the finest movements [22], such as the fingers of a hand [23], at a reasonable distance [24].…”