A multifaceted suite of metrics for comparative myoelectric prosthesis controller research

Abstract

Upper limb robotic (myoelectric) prostheses are technologically advanced, but challenging to use. In response, substantial research is being done to develop user-specific prosthesis controllers that can predict a person’s intended movements. Most studies that test and compare new controllers rely on simple assessment measures such as task scores (e.g., number of objects moved across a barrier) or duration-based measures (e.g., overall task completion time). These assessment measures, however, fail to capture valuable details about: the quality of device arm movements; whether these movements match users’ intentions; the timing of specific wrist and hand control functions; and users’ opinions regarding overall device reliability and controller training requirements. In this work, we present a comprehensive and novel suite of myoelectric prosthesis control evaluation metrics that better facilitates analysis of device movement details—spanning measures of task performance, control characteristics, and user experience. As a case example of their use and research viability, we applied these metrics in real-time control experimentation. Here, eight participants without upper limb impairment compared device control offered by a deep learning-based controller (recurrent convolutional neural network-based classification with transfer learning, or RCNN-TL) to that of a commonly used controller (linear discriminant analysis, or LDA). The participants wore a simulated prosthesis and performed complex functional tasks across multiple limb positions. Analysis resulting from our suite of metrics identified 16 instances of a user-facing problem known as the “limb position effect”. We determined that RCNN-TL performed the same as or significantly better than LDA in four such problem instances. We also confirmed that transfer learning can minimize user training burden. Overall, this study contributes a multifaceted new suite of control evaluation metrics, along with a guide to their application, for use in research and testing of myoelectric controllers today, and potentially for use in broader rehabilitation technologies of the future.

Myoelectric Prosthesis Control using Recurrent Convolutional Neural Network Regression Mitigates the Limb Position Effect

“…RCNNs have also proven to be advantageous as they can handle the intrinsic time-varying nature of muscle signals [24]. Our work capitalized on these benefits and established that position-aware RCNN controllers show promise towards mitigating the limb position effect [9]; as corroborated by our research that focused on the movements of participants without limb difference [9,[25][26][27].…”

“…(A) Myo armband on a participant’s forearm; (B) simulated prosthesis on a participant’s forearm, with labels indicating the sleeve, two pieces of liner, hand brace, distal ring, cushions, wrist motor, and hand motor; and (C) motion capture markers affixed to the simulated prosthesis, with the eight motion capture markers that remained attached to the hand circled, and the three additional individual markers for the ski pose calibration are labelled. Adapted from Williams et al [27]. …”

“…The clothespin movement sequence in RCRT Down follows the same order, but with each clothespin moved from vertical to horizontal locations. Adapted from Williams et al [27].…”

“…The Suite of Myoelectric Control Evaluation Metrics , introduced in our previous work [27], were used to compare prosthesis control resulting from RCNN-Class versus LDA-Baseline and RCNN-Reg versus LDA-Baseline. A summary of these metrics can be found in Table 1.…”

“…To identify occurrences where the limb position effect impeded control, we analyzed our resulting Control Characteristics metrics from the suite, as outlined in our previous work [27]—the thresholds used for identification of the limb position effect remained the same, with the total muscle activity metric eliminated from analysis thereof. Specifically, this present work’s limb position effect analysis considered trends across the Control Characteristics metrics’ medians and interquartile ranges (IQRs) for each task’s three movements, with larger medians and/or larger interquartile ranges providing evidence of degraded control.…”