Towards Robust Gesture Recognition by Characterizing the Sensing Quality of WiFi Signals

Abstract

WiFi-based gesture recognition emerges in recent years and attracts extensive attention from researchers. Recognizing gestures via WiFi signal is feasible because a human gesture introduces a time series of variations to the received raw signal. The major challenge for building a ubiquitous gesture recognition system is that the mapping between each gesture and the series of signal variations is not unique, exact the same gesture but performed at different locations or with different orientations towards the transceivers generates entirely different gesture signals (variations). To remove the location dependency, prior work proposes to use gesture-level location-independent features to characterize the gesture instead of directly matching the signal variation pattern. We observe that gesture-level features cannot fully remove the location dependency since the signal qualities inside each gesture are different and also depends on the location. Therefore, we divide the signal time series of each gesture into segments according to their qualities and propose customized signal processing techniques to handle them separately. To realize this goal, we characterize signal's sensing quality by building a mathematical model that links the gesture signal with the ambient noise, from which we further derive a unique metric i.e., error of dynamic phase index (EDP-index) to quantitatively describe the sensing quality of signal segments of each gesture. We then propose a quality-oriented signal processing framework that maximizes the contribution of the high-quality signal segments and minimizes the impact of low-quality signal segments to improve the performance of gesture recognition applications. We develop a prototype on COTS WiFi devices. The extensive experimental results demonstrate that our system can recognize gestures with an accuracy of more than 94% on average, and significant improvements compared with state-of-arts.

A Multitask Sign Language Recognition System Using Commodity Wi-Fi

“…Mobile Information Systems convolutional network was used to extract spatiotemporal information from six CSI action datasets. Reference [28] describes that the mapping relationship between gestures and CSI data is not unique, which difers from traditional gesture image data. Te CSI data generated by the same gesture can vary greatly by person, location, orientation, and scenarios.…”

“…Te CSI data generated by the same gesture can vary greatly by person, location, orientation, and scenarios. Gao et al [28] used dynamic phase index (EDP-index) error to remove the infuence of diferent positions and orientations on gestures to improve the quality of CSI-based wireless sensing. In [29], spatiotemporal information from CSI gesture data was extracted via a parallel long short-term memory fully convolutional network (LSTM-FCN) to accommodate user diferentiation and gesture diversity.…”

“…Reference [28] describes that the mapping relationship between gestures and the CSI data is not unique, which difers from the traditional gesture image data. Te CSI data generated by the same gesture can vary greatly by person, location, orientation, and scenarios.…”

“…Figure 7(a) shows that when only Lab276 samples are used as the training dataset, the recognition accuracy of Home276 is only 0.36 percent. According to references [4,26,28], the poor results may be attributed to the signifcant diference between the CSI data of the two environments. Inspired by the few-shot learning method [30], few samples from the target domain are added to the source domain.…”

Ubiquitous WiFi and Acoustic Sensing: Principles, Technologies, and Applications

“…Subsequently, the human behavior sensing technology based on WiFi signal was developed rapidly. The emergence of the CSI read interface makes CSI widely used in the WiFi sensing and sleep monitoring [3][4][5][6][7][8] , fall detection [9][10][11][12][13][14][15][16][17] , gesture detection [18][19][20][21][22][23][24][25][26][27][28][29][30][31][32] , lip language recognition [33,34] , crowd detection [35,36] , daily behavior detection [9,[37][38][39][40][41][42][43][44][45][46][47][48] , respiration and heartbeat detection [7,32,…”