A novel autonomous learning framework to enhance sEMG-based hand gesture recognition using depth information

Abstract

Hand gesture recognition using Surface Electromyography (sEMG) has been one of the most efficient motion analysis techniques in human-computer interaction in the last few decades. In particular, multichannel sEMG techniques have achieved stable performance in hand gesture recognition. However, the general solution of collecting and labeling large data manually leads to time-consuming implementation. A novel learning method is therefore needed to facilitate efficient data collection and preprocessing. In this paper, a novel autonomous learning framework is proposed to integrate the benefits of both depth vision and EMG signals, which automatically label the class of collected EMG data using depth information. It then utilizes a multiple layer neural network (MNN) classifier to achieve real-time recognition of the hand gestures using only the sEMG. The overall framework is demonstrated in an augmented reality application by the recognition of ten hand gestures using the Myo armband and an HTC VIVE PRO. The results show prominent

Toward Robust, Adaptiveand Reliable Upper-Limb Motion Estimation Using Machine Learning and Deep Learning–A Survey in Myoelectric Control

“…Despite the high correlation between sEMG and the intensity of neural drives to target muscles, sEMG signals alone may not be adequate enough for many practical applications of multi-functional upper-limb HMI, mainly because of 1) the large number of DoFs and non-cyclical nature of the upper extremity's movements [28]; 2) the complex patterns of EMG influenced by the anatomical and physiological properties of muscles, such as the limited spatial resolution caused by muscle cross-talk [27]. To this end, the fusion of sEMG with other signals have gained considerable attention, such that more complementary information can be obtained from the environment to compensate the shortcomings of sEMG.…”

“…Different from previous surveys, this paper provides a systematic review on recent progress towards model robustness, adaptation, and reliability in ML/DL based upper-limb myoelectric control. Firstly, the main factors that limit ML/DL implementations can be summarised as follows: 1) upper-limb movements are non-cyclical and have a large number of DoFs involved, whereas the information provided by sEMG signals may not be adequate enough for precise control [27,28]; 2) characteristics of sEMG are time-varying and user-specific, in the meantime they can be easily influenced by numerous disturbances in practical environments [22]; 3) high estimation accuracy can still lead to unintended activation, causing additional operations, cognitive burdens, and even unacceptable risks [20]. In this context, related efforts will be introduced accordingly in three aspects: 1) multi-modal fusion techniques to provide additional information in myoelectric control; 2) transfer learning methods to reduce domain shift impacts on ML/DL algorithms; and 3) post-processing approaches to enhance reliability of estimation outcomes.…”

A Novel SE-CNN Attention Architecture for sEMG-Based Hand Gesture Recognition

“…Therefore, it quickly catches the attention of scholars and is widely used in many aspects with high demands in gesture recognition. With high precision, easy wearing, and non-invasive properties, sEMG-based gesture recognition has become a hot spot in the research area [10][11][12].…”

A Novel Method for Hand Movement Recognition Based on Wavelet Packet Transform and Principal Component Analysis with Surface Electromyogram

“…Hand movements are the most meaningful and elementary form of human daily communication and represent the intentions expressed by people [ 1 , 2 ]. As a signal language input method, hand movement classification has important theoretical research significance and practical application value in the field of human-computer interaction [ 3 ]. As a result, hand movement classification technology that allows humans to communicate with computers more efficiently, conveniently, and naturally has developed into an important part of the field of artificial intelligence [ 4 , 5 ].…”