Unsupervised Domain Adversarial Self-Calibration for Electromyography-Based Gesture Recognition

Abstract

Surface electromyography (sEMG) provides an intuitive and non-invasive interface from which to control machines. However, preserving the myoelectric control system's performance over multiple days is challenging, due to the transient nature of the signals obtained with this recording technique. In practice, if the system is to remain usable, a time-consuming and periodic recalibration is necessary. In the case where the sEMG interface is employed every few days, the user might need to do this recalibration before every use. Thus, severely limiting the practicality of such a control method. Consequently, this paper proposes tackling the especially challenging task of unsupervised adaptation of sEMG signals, when multiple days have elapsed between each recording, by introducing Self-Calibrating Asynchronous Domain Adversarial Neural Network (SCADANN). SCADANN is compared with two state-of-the-art selfcalibrating algorithms developed specifically for deep learning within the context of EMG-based gesture recognition and three state-of-the-art domain adversarial algorithms. The comparison is made both on an offline and a dynamic dataset (20 participants per dataset), using two different deep network architectures with two different input modalities (temporal-spatial descriptors and spectrograms). Overall, SCADANN is shown to substantially and systematically improves classification performances over no recalibration and obtains the highest average accuracy for all tested cases across all methods.

Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future

“…Zhang J. et al ( 2019 ) developed DBN to deal with the non-linear and time-varying properties of sEMG signal, and proved the potential of DBN through the measured data. Although one of the advantages of unsupervised learning is that it eliminates the need for manual tagging, self-training strategy is likely to increase classification errors, especially with data distribution mutation (Huang et al, 2017 ; Côté-Allard et al, 2020 ).…”

“…TDNN processing of combined with sEMG and kinematics data could show excellent performance in the prediction of simultaneous motion of shoulder joint and elbow joint (Kwon and Kim, 2011 ; Blana et al, 2016 ; Day et al, 2020 ). Since the quantity of training datasets in DL affects the performance of the model, the limited datasets collected from multiple topics should be first extend by certain data-augmentation approaches that can also enhance the robustness of the model (Tsinganos et al, 2018 ; Yang et al, 2018 ; Côté-Allard et al, 2020 ). Moreover, due to the nature of the training in a neural network, the process of transfer learning is very straightforward (Yosinski et al, 2014 ; Côté-Allard et al, 2019 ).…”

Toward Long-Term FMG Model-Based Estimation of Applied Hand Force in Dynamic Motion During Human–Robot Interactions

“…Some of these studies addressed electrode shifts and day-to-day variability through adaptive transfer learning [24], [25], intersession gesture recognition using deep domain adaptation on unlabeled test data or fine-tuning labeled calibration data [26], and periodic recalibration for multiple days use for prosthetic control by applying transfer learning with few training data [27]. Furthermore, a recent study showed that aggregating source distributions from multiple users with deep transfer learning in gesture recognition enhanced model performance [28]. Since an FMG signal has similar characteristics, multiple domain adaptation was investigated with the traditional ML algorithm in this study.…”

Deep Cross-User Models Reduce the Training Burden in Myoelectric Control

“…In our prior work, the pooled-variance of subjects was used to create a subject-general model (one model trained and used by many subjects) that exceeded the accuracy of traditional within-subject models (one model for each subject). We have also validated it in the presence of confounding factors, such as inter-session and across day variations, using self-calibration (Côté-Allard et al, 2020b ). In these works, however, each of the test users supplied a full training set, resulting in little benefit to the training burden.…”