sEMG-Based Gesture Recognition With Embedded Virtual Hand Poses and Adversarial Learning

Abstract

To improve the accuracy of surface electromyography (sEMG)-based gesture recognition, we present a novel hybrid approach that combines real sEMG signals with corresponding virtual hand poses. The virtual hand poses are generated by means of a proposed cross-modal association model constructed based on the adversarial learning to capture the intrinsic relationship between the sEMG signals and the hand poses. We report comprehensive evaluations of the proposed approach for both frame-and window-based sEMG gesture recognitions on seven-sparse-multichannel and four-high-density-benchmark databases. The experimental results show that the proposed approach achieves significant improvements in sEMG-based gesture recognition compared to existing works. For frame-based sEMG gesture recognition, the recognition accuracy of the proposed framework is increased by an average of +5.2% on the sparse multichannel sEMG databases and by an average of +6.7% on the high-density sEMG databases compared to the existing methods. For window-based sEMG gesture recognition, the state-of-the-art recognition accuracies on three of the high-density sEMG databases are already higher than 99%, i.e., almost saturated; nevertheless, we achieve a +0.2% improvement. For the remaining eight sEMG databases, the average improvement with the proposed framework for the window-based approach is +2.5%. INDEX TERMSHand gesture recognition, surface electromyography (sEMG), myoelectric control, generative adversarial learning, virtual hand pose.

Action Recognition of Lower Limbs Based on Surface Electromyography Weighted Feature Method

“…The ending segment is ignored and not processed. The initial point is detected by using the signal frame energy method [25,26]. We use 64 sampling points of the signal as the sliding window frame and 32 sampling points as the increment and calculate the adaptive threshold th through the resting signal.…”

Hierarchical Attention-Based Astronaut Gesture Recognition: A Dataset and CNN Model

“…Many approaches have been explored in the field of gesture recognition. Some researches on gesture recognition attempt to use wearable devices such as data gloves [12] or electrodebased systems [13], [14]. However, this form of recognition is inconvenient and presents a limitation of human movement.…”

Deep Convolutional Generative Adversarial Network-Based EMG Data Enhancement for Hand Motion Classification

“… Anicet Zanini and Luna Colombini (2020) take DCGAN and neural style transfer to simulate each patient’s EMG tremor pattern with different frequencies and amplitudes under different sets of movements. Hu et al (2019) propose a two-step pipeline classification solution based on adversarial learning, achieving better gesture classification accuracy for both sparse multi-channel sEMG database and the high-density sEMG database. These studies address bio-signals as one-dimensional time series signals, and thus only one-dimensional convolutional layers are adopted.…”

“…use a GAN-based separation framework to separate the class-related EMG features for the detection of trunk compensatory patterns in stroke patients. Anicet Zanini and Luna Colombini (2020) take DCGAN and neural style transfer to simulate each patient's EMG tremor pattern with different frequencies and amplitudes under different sets of movements Hu et al (2019). propose a two-step pipeline classification solution based on adversarial learning, achieving better gesture classification accuracy for both sparse multichannel sEMG database and the high-density sEMG database.…”