Understanding the influence of confounding factors in myoelectric control for discrete gesture recognition

Abstract

Discrete myoelectric control-based gesture recognition has recently gained interest as a possible input modality for many emerging ubiquitous computing applications. Unlike the continuous control commonly employed in powered prostheses, discrete systems seek to recognize the dynamic sequences associated with gestures to generate event-based inputs. More akin to those used in general-purpose human-computer interaction, these could include, for example, a flick of the wrist to dismiss a phone call or a double tap of the index finger and thumb to silence an alarm. Moelectric control systems have been shown to achieve near-perfect classification accuracy, but in highly constrained offline settings. Real-world, online systems are subject to ``confounding factors'' (i.e., factors that hinder the real-world robustness of myoelectric control that are not accounted for during typical offline analyses), which inevitably degrade system performance, limiting their practical use. Although these factors have been widely studied in continuous prosthesis control, there has been little exploration of their impacts on discrete myoelectric control systems for emerging applications and use cases. Correspondingly, this work examines, for the first time, three confounding factors and their effect on the robustness of discrete myoelectric control: (1) limb position variability, (2) cross-day use, and a newly identified confound faced by discrete systems (3) gesture elicitation speed. Results from four different discrete myoelectric control architectures: (1) Majority Vote LDA, (2) Dynamic Time Warping, (3) an LSTM network trained with Cross Entropy, and (4) an LSTM network trained with Contrastive Learning, show that classification accuracy is significantly degraded (p<0.05) as a result of each of these confounds. This work establishes that confounding factors are a critical barrier that must be addressed to enable the real-world adoption of discrete myoelectric control for robust and reliable gesture recognition.

Big Data in Myoelectric Control: Large Multi-User Models Enable Robust Zero-Shot EMG-based Discrete Gesture Recognition

“…As described by (Saponas et al, 2008) and (Eddy et al, 2024b), the majority vote approach to discrete gesture recognition employs the legacy continuous classification approach but then converts the decision stream to a single discrete label by computing the mode of the predictions across all windows extracted from the entire gesture template. In this work, a linear discriminant analysis (LDA) classifier was used as the backbone for this majority vote approach, as it is commonly used in prosthesis control as a baseline condition for comparison (Botros et al, 2022;Duan et al, 2021).…”

“…Similar approaches have been previously adopted in EMG-based gesture recognition, such as for hand-writing (Li et al, 2013;Huang et al, 2010) and wake-gesture recognition (Kumar et al, 2021;Eddy et al, 2024a). It has also been recently shown that DTW-based approaches can even sometimes outperform deep-learning approaches when training user-dependent models for recognizing nine discrete gestures (Eddy et al, 2024b). Using the tslearn time-series implementation of DTW distance (Tavenard et al, 2020), the 1NN DTW was adopted as a baseline for time series approaches in this work.…”

“…Unlike the EMG-EPN612 dataset that was pre-segmented, this dataset was segmented using an automated active thresholding strategy using the "rest" class as a baseline. A further description of this dataset can be found in (Eddy et al, 2024b). Note that one participant of the original N=15 was excluded after it was found that the armband had been oriented in the wrong direction for their collection.…”

“…In this work, three different algorithmic approaches to achieving this discrete event-based control are explored: (1) Majority Vote LDA (MVLDA), a conventional machine learning approach; (2) Dynamic Time Warping (DTW), a common distance-based time series approach; and (3) Long Short-Term Memory (LSTM), a popular deep-learning time series approach. Additionally, a selection of EMG features -Root Mean Square (RMS), Mean Absolute Value (MAV), Wavelet Energy (WENG), Mean Power (MNP), Waveform Length (WL) -and feature sets -Hudgin's Time Domain (HTD) and Low Sampling 4 (LS4) -as made available through LibEMG, are explored (Eddy et al, 2023b). Unless otherwise specified, the RMS feature was used as the baseline for each model.…”

“…After splitting the discrete gesture into non-overlapping windows of 25ms (5 samples) and extracting the RMS feature, as done with the DTW model, the three LSTM layers projected the N windowed template to a single 128-dimensional point in the latent space. Note that all gestures were buffered with zeros to the maximum gesture size in the dataset, but the output at the actual end of the gesture was used to compute the loss (Eddy et al, 2024b). After passing through the LSTM layers, the one-dimensional embeddings were passed through three linear layers of 128, 64, and 6 neurons, and finally through a sigmoid function.…”