Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

Mendes, José Jair Alves; Freitas, Melissa La Banca; Campos, Daniel Prado; Farinelli, Felipe Adalberto; Stevan, Sérgio Luiz; Pichorim, Sérgio Francisco

doi:10.3390/s20164359

Cited by 15 publications

(9 citation statements)

References 63 publications

(104 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to Mendes [ 32 ], the sliding window length is selected as 1 s (containing 200 points for sEMG and 50 points for IMU) with a sliding step of 100 ms to segment the sEMG and IMU signals. Eventually, 601,489 sEMG samples of size 200 × 8 and 601,489 IMU samples of size 50 × 6 are generated.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Jiang

Song

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

Gesture recognition based on wearable devices is one of the vital components of human–computer interaction systems. Compared with skeleton-based recognition in computer vision, gesture recognition using wearable sensors has attracted wide attention for its robustness and convenience. Recently, many studies have proposed deep learning methods based on surface electromyography (sEMG) signals for gesture classification; however, most of the existing datasets are built for surface EMG signals, and there is a lack of datasets for multi-category gestures. Due to model limitations and inadequate classification data, the recognition accuracy of these methods cannot satisfy multi-gesture interaction scenarios. In this paper, a multi-category dataset containing 20 gestures is recorded with the help of a wearable device that can acquire surface electromyographic and inertial (IMU) signals. Various two-stream deep learning models are established and improved further. The basic convolutional neural network (CNN), recurrent neural network (RNN), and Transformer models are experimented on with our dataset as the classifier. The CNN and the RNN models’ test accuracy is over 95%; however, the Transformer model has a lower test accuracy of 71.68%. After further improvements, the CNN model is introduced into the residual network and augmented to the CNN-Res model, achieving 98.24% accuracy; moreover, it has the shortest training and testing time. Then, after combining the RNN model and the CNN-Res model, the long short term memory (LSTM)-Res model and gate recurrent unit (GRU)-Res model achieve the highest classification accuracy of 99.67% and 99.49%, respectively. Finally, the fusion of the Transformer model and the CNN model enables the Transformer-CNN model to be constructed. Such improvement dramatically boosts the performance of the Transformer module, increasing the recognition accuracy from 71.86% to 98.96%.

show abstract

Section: Methodsmentioning

confidence: 99%

“…In ref. [ 21 ], Nasri N took the Conv-GRU model as the classifier and created an sEMG-Controlled 3D game for rehabilitation therapies; moreover, the random forest (RF) model was also utilized by Mendes [ 32 ] to recognize Brazilian Sign Language in Sign Language recognition systems.…”

Section: Related Workmentioning

confidence: 99%

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Jiang

Song

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…As a result, different types of features and feature extraction methods have been adopted in sEMG signal processing. As summarized in Srisuwan et al (2018) and Mendes Junior et al (2020), time domain features such as Mean Absolute Value (MAV), Root Mean Square (RMS), and Variance (VAR) describe how sEMG signals vary temporally. They can be extracted in a simple but fast way and have been widely used in the study of sEMG signals (Hudgins et al, 1993;Englehart et al, 1999;Tkach et al, 2010;Samuel et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

“…On the other hand, frequency domain features are also used in the recognition and classification of sEMG signals, especially for the study of muscle fatigue (Phinyomark et al, 2012). However, in some cases they show relatively poor recognition results compared to using time-domain features (Srisuwan et al, 2018;Jong and Phukpattaranont, 2019;Mendes Junior et al, 2020). Furthermore, studies show better recognition performance by combining several types of features to form a new feature vector of the collected data (Atzori et al, 2016;Mendes Junior et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

A novel silent speech recognition approach based on parallel inception convolutional neural network and Mel frequency spectral coefficient

Zhang

Xie

et al. 2022

Front. Neurorobot.

View full text Add to dashboard Cite

Silent speech recognition breaks the limitations of automatic speech recognition when acoustic signals cannot be produced or captured clearly, but still has a long way to go before being ready for any real-life applications. To address this issue, we propose a novel silent speech recognition framework based on surface electromyography (sEMG) signals. In our approach, a new deep learning architecture Parallel Inception Convolutional Neural Network (PICNN) is proposed and implemented in our silent speech recognition system, with six inception modules processing six channels of sEMG data, separately and simultaneously. Meanwhile, Mel Frequency Spectral Coefficients (MFSCs) are employed to extract speech-related sEMG features for the first time. We further design and generate a 100-class dataset containing daily life assistance demands for the elderly and disabled individuals. The experimental results obtained from 28 subjects confirm that our silent speech recognition method outperforms state-of-the-art machine learning algorithms and deep learning architectures, achieving the best recognition accuracy of 90.76%. With sEMG data collected from four new subjects, efficient steps of subject-based transfer learning are conducted to further improve the cross-subject recognition ability of the proposed model. Promising results prove that our sEMG-based silent speech recognition system could have high recognition accuracy and steady performance in practical applications.

show abstract

“…Particularly, ANN algorithms have been extensively tested in EMG-based studies, which investigated various types of EMG data for a wide range of applications. For example, some studies tested both TD and FD features as input data [ 19 , 20 ], and others applied raw EMG signals directly to the ANN without a feature extraction process [ 21 , 22 ]. Additionally, the application of research was not limited to hand/finger gesture recognition but included the prediction of force load [ 23 , 24 ] and the detection of neuromuscular disorders [ 25 ].…”

Section: Introductionmentioning

confidence: 99%

Electromyogram-Based Classification of Hand and Finger Gestures Using Artificial Neural Networks

Lee

Min

Byun

2021

Sensors

View full text Add to dashboard Cite

Electromyogram (EMG) signals have been increasingly used for hand and finger gesture recognition. However, most studies have focused on the wrist and whole-hand gestures and not on individual finger (IF) gestures, which are considered more challenging. In this study, we develop EMG-based hand/finger gesture classifiers based on fixed electrode placement using machine learning methods. Ten healthy subjects performed ten hand/finger gestures, including seven IF gestures. EMG signals were measured from three channels, and six time-domain (TD) features were extracted from each channel. A total of 18 features was used to build personalized classifiers for ten gestures with an artificial neural network (ANN), a support vector machine (SVM), a random forest (RF), and a logistic regression (LR). The ANN, SVM, RF, and LR achieved mean accuracies of 0.940, 0.876, 0.831, and 0.539, respectively. One-way analyses of variance and F-tests showed that the ANN achieved the highest mean accuracy and the lowest inter-subject variance in the accuracy, respectively, suggesting that it was the least affected by individual variability in EMG signals. Using only TD features, we achieved a higher ratio of gestures to channels than other similar studies, suggesting that the proposed method can improve the system usability and reduce the computational burden.

show abstract

Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

Cited by 15 publications

References 63 publications

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

A novel silent speech recognition approach based on parallel inception convolutional neural network and Mel frequency spectral coefficient

Electromyogram-Based Classification of Hand and Finger Gestures Using Artificial Neural Networks

Contact Info

Product

Resources

About