Wi-Fringe: Leveraging Text Semantics in WiFi CSI-Based Device-Free Named Gesture Recognition

Abstract

The lack of adequate training data is one of the major hurdles in WiFi-based activity recognition systems. In this paper, we propose Wi-Fringe, which is a WiFi CSI-based device-free human gesture recognition system that recognizes named gestures, i.e., activities and gestures that have a semantically meaningful name in English language, as opposed to arbitrary free-form gestures. Given a list of activities (only their names in English text), along with zero or more training examples (WiFi CSI values) per activity, Wi-Fringe is able to detect all activities at runtime. We show for the first time that by utilizing the state-of-the-art semantic representation of English words, which is learned from datasets like the Wikipedia (e.g., Google's word-to-vector [1]) and verb attributes learned from how a word is defined (e.g, American Heritage Dictionary), we can enhance the capability of WiFi-based named gesture recognition systems that lack adequate training examples per class. We propose a novel cross-domain knowledge transfer algorithm between radio frequency (RF) and text to lessen the burden on developers and end-users from the tedious task of data collection for all possible activities. To evaluate Wi-Fringe, we collect data from four volunteers in a multi-person apartment and an office building for a total of 20 activities. We empirically quantify the trade-off between the accuracy and the number of unseen activities.

Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: A Review

“…In addition to the mentioned radar categories, several researchers have utilized other radio sensors to implement the phenomena of Radio Detection and Ranging. For example, Islam and Nirjon [35], and Pu et al [36] used the transmitted and the corresponding reflected WIFI signals to sense different gesture movements.…”

Enhancing RF Sensing with Deep Learning: A Layered Approach

“…We choose to introduce this layer before the deep learning layers as the neural network designs are driven by applications. [7] Sleep monitoring CNN+RNN Adversarial domain adaptation Zheng et al 2019 [5] Gesture recognition CNN+RNN HCF Fhager et al 2019 [8] Gesture recognition CNN Transfer learning Tamzeed et al 2019 [11] Gesture recognition CNN+LSTM Zero-shot learning Yang et al 2020 [12] Gesture recognition CNN+LSTM Teacher student network Wang et al 2019 [13] Pose estimation U-Net+attention Adversarial domain adaptation Guan et al 2020 [6] Imaging CNN GAN Yang et al 2019 [14] Channel prediction FNN Transfer learning…”

“…Deep learning can be employed to extract the spatial-temporal features from the RF data, then a detected activity will be chosen from a class of trained human activities. Recent RF-HAR applications include, among others, motion recognition [4], gesture recognition [5], [11], [12], [8] and sleep stage detection [7].…”

“…One of them is denoising, which removes noise and irrelevant information from an environment. Common approaches for denoising include filtering and Principal Component Analysis (PCA) [2], [11]. Filtering means separating signal and noise by transforming the RF signals into another domain.…”

SlideAugment: A Simple Data Processing Method to Enhance Human Activity Recognition Accuracy Based on WiFi

“…A study presented by Tsinghua University shows that in complex dynamic environment, the channel state information (CSI) from WiFi performs better than RSSI [ 6 ] in the field of indoor location. Since then, the channel state information from physical layer has been explored in activity recognition [ 7 ], indoor location [ 8 ], gesture recognition [ 9 , 10 ], and user identification [ 11 ]. The WiFi signals can be used to perform fall detection [ 12 ] and detect risky driving behavior [ 13 ].…”