A Novel Gesture Recognition System Based on CSI Extracted from a Smartphone with Nexmon Firmware

Abstract

In this paper, we propose a novel gesture recognition system based on a smartphone. Due to the limitation of Channel State Information (CSI) extraction equipment, existing WiFi-based gesture recognition is limited to the microcomputer terminal equipped with Intel 5300 or Atheros 9580 network cards. Therefore, accurate gesture recognition can only be performed in an area relatively fixed to the transceiver link. The new gesture recognition system proposed by us breaks this limitation. First, we use nexmon firmware to obtain 256 CSI subcarriers from the bottom layer of the smartphone in IEEE 802.11ac mode on 80 MHz bandwidth to realize the gesture recognition system’s mobility. Second, we adopt the cross-correlation method to integrate the extracted CSI features in the time and frequency domain to reduce the influence of changes in the smartphone location. Third, we use a new improved DTW algorithm to classify and recognize gestures. We implemented vast experiments to verify the system’s recognition accuracy at different distances in different directions and environments. The results show that the system can effectively improve the recognition accuracy.

Human Activity Recognition With Commercial WiFi Signals

“…This paper compared the recognition methods of other articles. In [10], a WiFi-based system was also used for human activity recognition. Six specific actions were identified by collecting the amplitude information of CSI.…”

Research Progress of Human–Computer Interaction Technology Based on Gesture Recognition

“…They combined an effective denoising method to filter environmental interference and efficiently selected optimal subcarriers, thereby reducing system computation costs. Li Tao et al [24], utilizing the nexmon firmware, obtained 256 CSI subcarriers from the underlying layer of smartphones operating in IEEE 802.11 ac mode with an 80 MHz bandwidth. They then fused the extracted CSI features in the time and frequency domains using a cross−correlation method and ultimately recognized gestures using an improved DTW algorithm, breaking the limitation that gesture recognition can only be performed within relatively fixed regions along the transmission link.…”

Private Data Exfiltration from Cyber-Physical Systems Using Channel State Information

“…As RSS is the combination of many subcarriers, this gives a fairly poor resolution when examining the environments effect on the channel state. CSI is conventionally used to improve the reliability of communication, but being able to view subcarrier level effects caused by the environment has led to improvements in the fields of localization [41,43,47], device-free activity recognition [13,28,48], fall detection and health monitoring [46], smartphone gesture recognition [30], and cross-technology communication [16].…”