Evaluating Electromyography and Sonomyography Sensor Fusion to Estimate Lower-Limb Kinematics Using Gaussian Process Regression

Abstract

Research on robotic lower-limb assistive devices over the past decade has generated autonomous, multiple degree-of-freedom devices to augment human performance during a variety of scenarios. However, the increase in capabilities of these devices is met with an increase in the complexity of the overall control problem and requirement for an accurate and robust sensing modality for intent recognition. Due to its ability to precede changes in motion, surface electromyography (EMG) is widely studied as a peripheral sensing modality for capturing features of muscle activity as an input for control of powered assistive devices. In order to capture features that contribute to muscle contraction and joint motion beyond muscle activity of superficial muscles, researchers have introduced sonomyography, or real-time dynamic ultrasound imaging of skeletal muscle. However, the ability of these sonomyography features to continuously predict multiple lower-limb joint kinematics during widely varying ambulation tasks, and their potential as an input for powered multiple degree-of-freedom lower-limb assistive devices is unknown. The objective of this research is to evaluate surface EMG and sonomyography, as well as the fusion of features from both sensing modalities, as inputs to Gaussian process regression models for the continuous estimation of hip, knee and ankle angle and velocity during level walking, stair ascent/descent and ramp ascent/descent ambulation. Gaussian process regression is a Bayesian nonlinear regression model that has been introduced as an alternative to musculoskeletal model-based techniques. In this study, time-intensity features of sonomyography on both the anterior and posterior thigh along with time-domain features of surface EMG from eight muscles on the lower-limb were used to train and test subject-dependent and task-invariant Gaussian process regression models for the continuous estimation of hip, knee and ankle motion. Overall, anterior sonomyography sensor fusion with surface EMG significantly improved estimation of hip, knee and ankle motion for all ambulation tasks (level ground, stair and ramp ambulation) in comparison to surface EMG alone. Additionally, anterior sonomyography alone significantly improved errors at the hip and knee for most tasks compared to surface EMG. These findings help inform the implementation and integration of volitional control strategies for robotic assistive technologies.

Exoskeletons: a review of recent progress

“…Workers from the University of Texas at Austin have developed a lower-limb exoskeleton control technique for user-intent recognition, which combines surface EMG with sonomyography (Rabe and Fey, 2022). Sonomyography is the real-time dynamic ultrasound imaging of skeletal muscle, but in contrast to EMG, its ability to predict multiple lower-limb joint kinematics during ambulation tasks and its potential as an input for multiple DOF assistive devices is unknown.…”

“…Schematic overview of the method. Data collection during five ambulation modes, sensing modality feature generation, regression model implementation and ultimately hip, knee and ankle joint kinematic prediction (Credit: Rabe & Fey, 2022 and, Frontiers in Robotics and AI . doi: 10.3389/frobt.2022.716545)…”

Clinical viability of magnetic bead implants in muscle

“…While significant advances have been made in robotic control via sEMG ( Farina et al, 2017 ), even myoelectric signals measured using invasive strategies ( Weir et al, 2008 ) cannot fully communicate intended muscle forces or joint movements without also incorporating measurements of muscle length and speed ( Zajac 1989 ). Recent work has suggested the use of ultrasound combined with sEMG to improve upon robotic position control ( Zhang et al, 2021 ; Rabe and Fey 2022 ). However, a real-time sensing technology has not yet been developed that can directly and reliably measure muscle length and speed in humans.…”

A-Mode Ultrasound-Based Prediction of Transfemoral Amputee Prosthesis Walking Kinematics via an Artificial Neural Network

“…Gesture recognition via B-mode ultrasound has been successfully integrated in the control of upper-limb powered prostheses [38]. In lower-limb applications with able-bodied subjects, B-mode ultrasound sensing has allowed for the continuous classification of ambulation modes [34] and prediction of joint kinematics [39] and kinetics [40]. Furthermore, muscle fatigue [41] and muscle force [42] measurements from B-mode ultrasound have been used to determine exoskeleton assistance.…”