Robust Real-Time Embedded EMG Recognition Framework Using Temporal Convolutional Networks on a Multicore IoT Processor

Zanghieri, Marcello; Benatti, Simone; Burrello, Alessio; Kartsch, Victor; Conti, Francesco; Benini, Luca

doi:10.1109/tbcas.2019.2959160

Cited by 110 publications

(121 citation statements)

References 42 publications

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…At present, the application of deep learning in the field of myoelectric control mostly relies on discrete estimation [8] [9] [34] [35]. Although TCN can achieve good results in discrete estimation [8], it cannot be effectively applied to continuous estimates. Researchers are trying to find effective methods to estimate the continuous movement of the hand.…”

Section: Discussionmentioning

confidence: 99%

Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electromyographic signals

Guo

Wang

et al. 2021

J. Neural Eng.

View full text Add to dashboard Cite

Objective. Estimation of finger kinematics is an important function of an intuitive humanmachine interface, such as gesture recognition. Here, we propose a novel deep learning method, named Long Exposure Convolutional Memory Network (LE-ConvMN), and use it to proportionally estimate finger joint angles through surface electromyographic (sEMG) signals. Approach. We use a convolution structure to replace the neuron structure of traditional Long Short-Term Memory (LSTM) networks, and use the long exposure data structure which retains the spatial and temporal information of the electrodes as input. The Ninapro database, which contains continuous finger gestures and corresponding sEMG signals was used to verify the efficiency of the proposed deep learning method. The proposed method was compared with LSTM and Sparse Pseudo-input Gaussian Process (SPGP) on this database to predict the 10 main joint angles on the hand based on sEMG. The correlation coefficient (CC) was evaluated using the three methods on eight healthy subjects, and all the methods adopted the root mean square (RMS) features. Main results. The experimental results showed that the average CC, RMSE, NRMSE of the proposed LE-ConvMN method (0.82±0.03,11.54±1.89,0.12±0.013) was significantly higher than SPGP (0.65±0.05, p＜ 0.001; 15.51±2.82, p＜0.001; 0.16±0.01, p＜0.001) and LSTM (0.64±0.06, p＜0.001; 14.77±3.21, p＜0.001; 0.15±0.02, p=＜0.001). Furthermore, the proposed real-timeestimation method has a computation cost of only approximately 82 ms to output one state of ten joints (average value of 10 tests on TitanV GPU). Significance. The proposed LE-ConvMN method could efficiently estimate the continuous movement of fingers with sEMG, and its performance is significantly superior to two established deep learning methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electromyographic signals

Guo

Wang

et al. 2021

J. Neural Eng.

View full text Add to dashboard Cite

show abstract

“…A 20–450 Hz digital band pass filter and amplitude normalization were operated to improve the signal-to-noise ratio (SNR) of the sEMG signal. In view of the real-time application of the proposed method, the total duration for collection, preprocessing, classification, and robotic arm control of sEMG signals was restricted within an upper limit of 300 ms [ 43 – 46 ]. The sEMG data were segmented into 192 ms analysis windows with 15 ms of slippage rate including offline/online classification and real-time control.…”

Section: Methodsmentioning

confidence: 99%

“…A commercial robotic arm was also tested to preliminarily validate the real-time control performance of the proposed deep-learning model. The entire control course took 269 ms, satisfying the requirements for real-time control within 300 ms [ 43 – 46 ].…”

Section: Introductionmentioning

confidence: 99%

Lw-CNN-Based Myoelectric Signal Recognition and Real-Time Control of Robotic Arm for Upper-Limb Rehabilitation

Guo

Yang

et al. 2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Deep-learning models can realize the feature extraction and advanced abstraction of raw myoelectric signals without necessitating manual selection. Raw surface myoelectric signals are processed with a deep model in this study to investigate the feasibility of recognizing upper-limb motion intents and real-time control of auxiliary equipment for upper-limb rehabilitation training. Surface myoelectric signals are collected on six motions of eight subjects’ upper limbs. A light-weight convolutional neural network (Lw-CNN) and support vector machine (SVM) model are designed for myoelectric signal pattern recognition. The offline and online performance of the two models are then compared. The average accuracy is (90 ± 5)% for the Lw-CNN and (82.5 ± 3.5)% for the SVM in offline testing of all subjects, which prevails over (84 ± 6)% for the online Lw-CNN and (79 ± 4)% for SVM. The robotic arm control accuracy is (88.5 ± 5.5)%. Significance analysis shows no significant correlation ( p = 0.056) among real-time control, offline testing, and online testing. The Lw-CNN model performs well in the recognition of upper-limb motion intents and can realize real-time control of a commercial robotic arm.

show abstract

“…However, similar to sEMG, FMG also has a downside caused by stochastic signal variation within the same class of gestures along time lasting, leading to low inter-session classification performance [23,24]. To overcome this problem, usually feature engineering and sophisticated machine learning algorithms are employed [25,26].…”

Section: Introductionmentioning

confidence: 99%

A Machine Learning Processing Pipeline for Reliable Hand Gesture Classification of FMG Signals with Stochastic Variance

Asfour

Menon

Jiang

2021

Sensors

View full text Add to dashboard Cite

ForceMyography (FMG) is an emerging competitor to surface ElectroMyography (sEMG) for hand gesture recognition. Most of the state-of-the-art research in this area explores different machine learning algorithms or feature engineering to improve hand gesture recognition performance. This paper proposes a novel signal processing pipeline employing a manifold learning method to produce a robust signal representation to boost hand gesture classifiers’ performance. We tested this approach on an FMG dataset collected from nine participants in 3 different data collection sessions with short delays between each. For each participant’s data, the proposed pipeline was applied, and then different classification algorithms were used to evaluate the effect of the pipeline compared to raw FMG signals in hand gesture classification. The results show that incorporating the proposed pipeline reduced variance within the same gesture data and notably maximized variance between different gestures, allowing improved robustness of hand gestures classification performance and consistency across time. On top of that, the pipeline improved the classification accuracy consistently regardless of different classifiers, gaining an average of 5% accuracy improvement.

show abstract

Robust Real-Time Embedded EMG Recognition Framework Using Temporal Convolutional Networks on a Multicore IoT Processor

Cited by 110 publications

References 42 publications

Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electromyographic signals

Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electromyographic signals

Lw-CNN-Based Myoelectric Signal Recognition and Real-Time Control of Robotic Arm for Upper-Limb Rehabilitation

A Machine Learning Processing Pipeline for Reliable Hand Gesture Classification of FMG Signals with Stochastic Variance

Contact Info

Product

Resources

About