Hand Gesture Recognition for an Off-the-Shelf Radar by Electromagnetic Modeling and Inversion

Sluÿters, Arthur; Lambot, Sébastien; Vanderdonckt, Jean

doi:10.1145/3490099.3511107

Cited by 18 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2. If additional gestures are required, modifications to our basic gesture recognition pipeline may be necessary, such as preprocessing the raw signal or implementing new recognition techniques [29]. Fig.…”

Section: Implementation Approach and Prototypementioning

confidence: 99%

SkySculptor: Intuitive Drone Control Through Ground-Integrated Radar and Foot Gestures in Smart Indoor Environments

Siean

2024

IJACSA

View full text Add to dashboard Cite

SkySculptor is a software application designed to optimize drone control in smart indoor environments. The primary focus is on using gesture input for drone control, particularly investigating mid-air free-foot interactions detected by radar sensing. This software application simplifies the process of controlling drones in smart indoor environments. Additionally, outcomes of utilizing a 15-antenna ultra-wideband 3D radar are presented, establishing a dictionary of six directional swipe gestures for controlling drone functions. Based on the findings of this research article, guidelines for the future development of software applications for drone control in intelligent indoor environments are proposed.

show abstract

Section: Implementation Approach and Prototypementioning

confidence: 99%

SkySculptor: Intuitive Drone Control Through Ground-Integrated Radar and Foot Gestures in Smart Indoor Environments

Siean

2024

IJACSA

View full text Add to dashboard Cite

show abstract

“…Prior works on radar-based gesture recognition [14], [18], [38], [39] mostly relied on a fixed, custom radar and on advanced ML/DL algorithms to cope with the complexity of radar signals. More recent works support dynamic, real-time recognition in mobile contexts of use [15], [18], [40]. The Google Soli chip [41] is embedded in a smartphone for recognizing various classes of gestures.…”

Section: A Radar-based Gesture Interaction 1) Performancementioning

confidence: 99%

Analysis of User-Defined Radar-Based Hand Gestures Sensed Through Multiple Materials

Sluÿters,

Lambot,

Vanderdonckt

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

Radar sensing can penetrate non-conducting materials, such as glass, wood, and plastic, which makes it appropriate for recognizing gestures in environments with poor visibility, limited accessibility, and privacy sensitivityWhile the performance of radar-based gesture recognition in these environments has been extensively researched, the preferences that users express for these gestures are less known. To analyze such gestures simultaneously according to their user preference and their system recognition performance, we conducted three gesture elicitation studies each with n 1 =30 participants to identify user-defined, radar-based gestures sensed through three distinct materials: the glass of a shop window, the wood of an office door, and polyvinyl chloride in an emergency. On this basis, we created a new dataset of nine selected gesture classes for n 2 =20 participants repeating twice the same gesture captured by radar through three materials, i.e., glass, wood, and polyvinyl chloride. To uniformly compare recognition rates in these conditions with sensing variations, a specifically tailored procedure was defined and conducted with one-shot radar calibration to train and evaluate a gesture recognizer. 'Wood' achieved the best recognition rate (96.44%), followed by 'Polyvinyl chloride' and 'Glass'. We perform a preference-performance analysis of the gestures by combining the agreement rate from the elicitation studies and the recognition rate from the evaluation.

show abstract

“…[ 74 , 75 , 76 , 77 , 78 , 79 , 80 ], etc. [ 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 ]. The originality of the selected scientific studies is highlighted below.…”

Section: Related Workmentioning

confidence: 99%

“…In Ref. [ 89 ], the authors showed the feasibility of hand gesture recognition using electromagnetic waves and machine learning. The authors in Ref.…”

Section: Related Workmentioning

confidence: 99%

Audio-Visual Speech and Gesture Recognition by Sensors of Mobile Devices

Ryumin

Ivanko

Ryumina

2023

Sensors

View full text Add to dashboard Cite

Audio-visual speech recognition (AVSR) is one of the most promising solutions for reliable speech recognition, particularly when audio is corrupted by noise. Additional visual information can be used for both automatic lip-reading and gesture recognition. Hand gestures are a form of non-verbal communication and can be used as a very important part of modern human–computer interaction systems. Currently, audio and video modalities are easily accessible by sensors of mobile devices. However, there is no out-of-the-box solution for automatic audio-visual speech and gesture recognition. This study introduces two deep neural network-based model architectures: one for AVSR and one for gesture recognition. The main novelty regarding audio-visual speech recognition lies in fine-tuning strategies for both visual and acoustic features and in the proposed end-to-end model, which considers three modality fusion approaches: prediction-level, feature-level, and model-level. The main novelty in gesture recognition lies in a unique set of spatio-temporal features, including those that consider lip articulation information. As there are no available datasets for the combined task, we evaluated our methods on two different large-scale corpora—LRW and AUTSL—and outperformed existing methods on both audio-visual speech recognition and gesture recognition tasks. We achieved AVSR accuracy for the LRW dataset equal to 98.76% and gesture recognition rate for the AUTSL dataset equal to 98.56%. The results obtained demonstrate not only the high performance of the proposed methodology, but also the fundamental possibility of recognizing audio-visual speech and gestures by sensors of mobile devices.

show abstract

Hand Gesture Recognition for an Off-the-Shelf Radar by Electromagnetic Modeling and Inversion

Cited by 18 publications

References 41 publications

SkySculptor: Intuitive Drone Control Through Ground-Integrated Radar and Foot Gestures in Smart Indoor Environments

SkySculptor: Intuitive Drone Control Through Ground-Integrated Radar and Foot Gestures in Smart Indoor Environments

Analysis of User-Defined Radar-Based Hand Gestures Sensed Through Multiple Materials

Audio-Visual Speech and Gesture Recognition by Sensors of Mobile Devices

Contact Info

Product

Resources

About