Toward Real-Time Muscle Force Inference and Device Control via Optical-Flow-Tracked Muscle Deformation

Abstract

Despite the utility of musculoskeletal dynamics modeling, there exists no safe, noninvasive method of measuring in vivo muscle output force in real time -limiting both biomechanical insight into dexterous motion and intuitive control of assistive devices. In this paper, we demonstrate that muscle deformation constitutes a promising, yet unexplored signal from which to 1) infer such forces and 2) build novel device control schemes. Through a case study of the elbow joint on a preliminary cohort of 10 subjects, we show that muscle deformation (specifically, thickness change of the brachioradialis, as measured via ultrasound and tracked via optical flow) correlates well with elbow output force to an extent comparable with standard surface electromyography (sEMG) activation during varied isometric elbow contraction. We then show that, given real-time visual feedback, subjects can readily perform a trajectory tracking task using this deformation signal, and that they largely prefer this method to a comparable sEMG-based control scheme and perform the tracking task with similar accuracy. Together, these contributions illustrate muscle deformation's potential utility for both biomechanical study of individual muscle dynamics and device control, in a manner that -thanks to, unlike sEMG, the localized nature of the signal and its tight mechanistic coupling to output force -is readily extensible to multiple muscles and device degrees of freedom. To enable such future extensions, all modeling, tracking, and visualization software described in this paper, as well as all raw and processed data, have been made available on SimTK as part of the OpenArm project (https://simtk.org/projects/openarm) for general research use.

Intelligent Systems for Muscle Tracking: A Review on Sensor‐Algorithm Synergy

“…However, sEMGs are often preferred due to both their high temporal resolution and ability to detect isometric contractions. However, Hallock et al [ 180 ] developed an Ultrasound‐based system utilizing the standard iterative Lucas–Kanade method of optical flow estimation to track muscle thickness and correlate it with force output. While the system was not wearable, its capability to correlate muscle thickness to muscle output force for isometric contractions is particularly unique in the field of ultrasound development and is of particular interest for rehabilitative purposes.…”