Exploiting the Intertemporal Structure of the Upper-Limb sEMG: Comparisons between an LSTM Network and Cross-Sectional Myoelectric Pattern Recognition Methods

Olsson, Alexander E.; Malešević, Nebojša; Björkman, Anders; Antfolk, Christian

doi:10.1109/embc.2019.8856648

Cited by 15 publications

(18 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(i) they are cross-sectional [18][19][20][21]; that is, they cannot extract the inter-temporal dependencies that exist between feature extraction windows;…”

Section: Introductionmentioning

confidence: 99%

“…These limitations are depicted in the left side of figure 1. Therefore, the development of feature extraction methods that can capture the temporal dynamics of the EMG signals in a spatially-aware way, such as the one shown on the right side of figure 1, has received increased attention recently [15,21,[25][26][27][28][29][30][31][32][33][34][35][36][37][38].…”

Section: Introductionmentioning

confidence: 99%

“…To address the former, many hand-crafted features [15,[25][26][27] and deep learningbased approaches [21,[28][29][30] are introduced. The fusion of time-domain descriptors (fTDD) and deep Long Short-Term Memory (LSTM) networks yield higher classification accuracy than the conventional time-domain (TD) features [19,21,39,40]. These methods excel in extracting the temporal information by concatenating features.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Decoding HD-EMG Signals for Myoelectric Control - How Small Can the Analysis Window Size be?

Khushaba

Nazarpour

2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Objective: The efficacy of an adopted feature extraction method directly affects the classification of the electromyographic (EMG) signals in myoelectric control applications. Most methods attempt to extract the dynamics of the multi-channel EMG signals in the time domain and on a channel-by-channel, or at best pairs of channels, basis. However, considering multi-channel information to build a similarity matrix has not been taken into account.Approach: Combining methods of long and short-term memory (LSTM) and dynamic temporal warping (DTW), we developed a new feature, called spatio-temporal warping (STW), for myoelectric signals. This method captures the spatio-temporal relationships of multi-channels EMG signals.Main Results: Across four online databases, we show that in terms of average classification error and standard deviation values, the STW feature outperforms traditional features by 5% to 17%. In comparison to the more recent deep learning models, e.g. convolutional neural networks (CNN), STW outperformed by 5% to 18%. Also, STW showed enhanced performance when compared to the CNN+LSTM model by 2% to 14%. All differences were statistically significant with a large effect size.Significance: This feasibility study provides evidence supporting the hypothesis that spatio-temporal warping of the EMG signals can enhance the classification accuracy in an explainable way when compared to recent deep learning methods. Future work includes real-time implementation of the method and testing for prosthesis control.

show abstract

“…(i) they are cross-sectional [18][19][20][21]; that is, they cannot extract the inter-temporal dependencies that exist between feature extraction windows;…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Decoding HD-EMG Signals for Myoelectric Control - How Small Can the Analysis Window Size be?

Khushaba

Nazarpour

2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…With mechanical delays inherent to the prosthesis itself, this leaves approximately 125 ms available for algorithm-induced delays ( Farrell and Weir, 2007 ). To circumvent these issues, a possibility is to let LSTM models operate continuously on iEMG samples as they are acquired instead of on a window-wise basis, as has been considered in previous studies using sEMG ( Olsson et al, 2019 ). Nevertheless, it is apparent that future work that focuses on finding more computationally efficient LSTM regression architectures would be of value.…”

Section: Discussionmentioning

confidence: 99%

End-to-End Estimation of Hand- and Wrist Forces From Raw Intramuscular EMG Signals Using LSTM Networks

et al. 2021

View full text Add to dashboard Cite

Processing myoelectrical activity in the forearm has for long been considered a promising framework to allow transradial amputees to control motorized prostheses. In spite of expectations, contemporary muscle–computer interfaces built for this purpose typically fail to satisfy one or more important desiderata, such as accuracy, robustness, and/or naturalness of control, in part due to difficulties in acquiring high-quality signals continuously outside laboratory conditions. In light of such problems, surgically implanted electrodes have been made a viable option that allows for long-term acquisition of intramuscular electromyography (iEMG) measurements of spatially precise origin. As it stands, the question of how information embedded in such signals is best extracted and combined across multiple channels remains open. This study presents and evaluates an approach to this end that uses deep neural networks based on the Long Short-Term Memory (LSTMs) architecture to regress forces exerted by multiple degrees of freedom (DoFs) from multichannel iEMG. Three deep learning models, representing three distinct regression strategies, were evaluated: (I) One-to-One, wherein each DoF is separately estimated by an LSTM model processing a single iEMG channel, (II) All-to-One, wherein each DoF is separately estimated by an LSTM model processing all iEMG channels, and (III) All-to-All, wherein a single LSTM model with access to all iEMG channels estimates all DoFs simultaneously. All models operate on raw iEMG, with no preliminary feature extraction required. When evaluated on a dataset comprising six iEMG channels with concurrent force measurements acquired from 14 subjects, all LSTM strategies were found to significantly outperform a baseline feature-based linear control regression method. This finding indicates that recurrent neural networks can learn to transform raw forearm iEMG signals directly into representations that correlate with forces exerted at the level of the hand to a greater degree than simple features do. Furthermore, the All-to-All and All-to-One strategies were found to exhibit better performance than the One-to-One strategy. This finding suggests that, in spite of the spatially local nature of signals, iEMG from muscles not directly actuating the relevant DoF can provide contextual information that aid in decoding motor intent.

show abstract

“…[19][20][21][22][23]) and recurrent neural networks (e.g. [24,25]), has recently found widespread use in myoelectric control research [26] and has frequently attained exceptional accuracy scores. Unlike their 'classical' machine learning counterparts, such methods avert the need for manual feature engineering via their ability to gainfully operate directly on raw sEMG, but are often hampered by a need for time-consuming hyperparameter tuning; large datasets; and/or requirement on computational resources infeasible for embedded systems [27].…”

mentioning

confidence: 99%

Learning regularized representations of categorically labelled surface EMG enables simultaneous and proportional myoelectric control

Olsson

Malešević

Björkman

et al. 2021

J NeuroEngineering Rehabil

Self Cite

View full text Add to dashboard Cite

Background Processing the surface electromyogram (sEMG) to decode movement intent is a promising approach for natural control of upper extremity prostheses. To this end, this paper introduces and evaluates a new framework which allows for simultaneous and proportional myoelectric control over multiple degrees of freedom (DoFs) in real-time. The framework uses multitask neural networks and domain-informed regularization in order to automatically find nonlinear mappings from the forearm sEMG envelope to multivariate and continuous encodings of concurrent hand- and wrist kinematics, despite only requiring categorical movement instruction stimuli signals for calibration. Methods Forearm sEMG with 8 channels was collected from healthy human subjects (N = 20) and used to calibrate two myoelectric control interfaces, each with two output DoFs. The interfaces were built from (I) the proposed framework, termed Myoelectric Representation Learning (MRL), and, to allow for comparisons, from (II) a standard pattern recognition framework based on Linear Discriminant Analysis (LDA). The online performances of both interfaces were assessed with a Fitts’s law type test generating 5 quantitative performance metrics. The temporal stabilities of the interfaces were evaluated by conducting identical tests without recalibration 7 days after the initial experiment session. Results Metric-wise two-way repeated measures ANOVA with factors method (MRL vs LDA) and session (day 1 vs day 7) revealed a significant ($$p<0.05$$ p < 0.05 ) advantage for MRL over LDA in 5 out of 5 performance metrics, with metric-wise effect sizes (Cohen’s $$d$$ d ) separating MRL from LDA ranging from $$\left|d\right|=0.62$$ d = 0.62 to $$\left|d\right|=1.13$$ d = 1.13 . No significant effect on any metric was detected for neither session nor interaction between method and session, indicating that none of the methods deteriorated significantly in control efficacy during one week of intermission. Conclusions The results suggest that MRL is able to successfully generate stable mappings from EMG to kinematics, thereby enabling myoelectric control with real-time performance superior to that of the current commercial standard for pattern recognition (as represented by LDA). It is thus postulated that the presented MRL approach can be of practical utility for muscle-computer interfaces.

show abstract

Exploiting the Intertemporal Structure of the Upper-Limb sEMG: Comparisons between an LSTM Network and Cross-Sectional Myoelectric Pattern Recognition Methods

Cited by 15 publications

References 18 publications

Decoding HD-EMG Signals for Myoelectric Control - How Small Can the Analysis Window Size be?

Decoding HD-EMG Signals for Myoelectric Control - How Small Can the Analysis Window Size be?

End-to-End Estimation of Hand- and Wrist Forces From Raw Intramuscular EMG Signals Using LSTM Networks

Learning regularized representations of categorically labelled surface EMG enables simultaneous and proportional myoelectric control

Contact Info

Product

Resources

About