Optimal Reconstruction of Human Motion From Scarce Multimodal Data

Abstract

Wearable sensing has emerged as a promising solution for enabling unobtrusive and ergonomic measurements of the human motion. However, the reconstruction performance of these devices strongly depends on the quality and the number of sensors, which are typically limited by wearability and economic constraints. A promising approach to minimize the number of sensors is to exploit dimensionality reduction approaches that fuse prior information with insufficient sensing signals, through minimum variance estimation. These methods were successfully used for static hand pose reconstruction, but their translation to motion reconstruction has not been attempted yet. In this work, we propose the usage of functional principal component analysis to decompose multimodal, time-varying motion profiles in terms of linear combinations of basis functions. Functional decomposition enables the estimation of the a priori covariance matrix, and hence the fusion of scarce and noisy measured data with a priori information. We also consider the problem of identifying which elemental variables to measure as the most informative for a given class of tasks. We applied our method to two different datasets of upper limb motion D1 (joint trajectories) and D2 (joint trajectories + EMG data) considering an optimal set of measures (four joints for D1 out of seven, three joints, and eight EMGs for D2 out of seven and twelve, respectively). We found that our approach enables the reconstruction of upper limb motion with a median error of 0.013 ± 0.006 rad for D1 (relative median error 0.9%), and 0.038 ± 0.023 rad and 0.003 ± 0.002 mV for D2 (relative median error 2.9% and 5.1%, respectively).

A Multi-Modal Under-Sensorized Wearable System for Optimal Kinematic and Muscular Tracking of Human Upper Limb Motion

“…Additionally, it is hard to develop a trustworthy estimation of the covariance matrix from heterogeneous data due to the concurrent reconstruction of multimodal motion-related data (such as joint angles and EMG signals) [ 33 ]. In [ 34 ], we proposed to generalize these methods for the estimation of multi-modal time-varying data of the upper limb. The method built upon the existence of covariation patterns in human upper limb motions, as we demonstrated in [ 23 ] and the usage of functional analysis for reconstructing the whole trajectory over time and estimating the covariance matrix.…”

“…However, in [ 34 ], the analysis was performed assuming as state variables the joint angular values and the muscle envelopes, while the non-linear mapping between sensors and state variables was not considered. In this paper, we build upon our previous work and extend the method to design an under-sensorized wearable system for multimodal acquisition of human upper limb trajectories.…”

“…We assume to have at disposal IMUs for kinematic recording and surface sEMGs for muscular activity acquisition, and that their number is not in a bijective relation with all the DoFs used to describe the whole muscle-skeletal status. We generalize the optimal sensing setup identified in [ 34 ] to the more challenging case in which one sensor may record the activity of multiple DoFs. Indeed, since the goal is now to reduce the number of employed sensor elements, instead of selecting the single optimal degrees of freedom, i.e., the ones that are associated with a reduced estimation uncertainty, our targeted optimal joint angles are those that enable a compromise between optimal reconstruction and the minimization of the sensing resource in use.…”

“…To target both objectives, we select as measures the shoulder joints. In this way, we minimize the differences with respect to the optimal setup reported in [ 34 ]. Finally, we build a real prototype of an optimal under-sensorized setup for upper limbs (i.e., which has a number of elements lower than the number required to measure all states of the system), with only two IMUs to retrieve angles from the shoulder by implementing an Unscented Kalman Filter (UKF).…”

“…Finally, we build a real prototype of an optimal under-sensorized setup for upper limbs (i.e., which has a number of elements lower than the number required to measure all states of the system), with only two IMUs to retrieve angles from the shoulder by implementing an Unscented Kalman Filter (UKF). We integrated these measurements with the optimal sEMGs identified in [ 34 ], discarding the other ones, and using a commercially available fully sensorized solution (i.e., Xsens) to have a ground truth for result comparison. Extensive tests on a dataset collected with our framework demonstrate that our method can effectively compensate for missing recordings (corresponding to two out of five joint angles and four out of twelve sEMG signals), with minimum impact on the estimation error, achieving a median normalized RMS error of on the non-measured joints and of on the non-measured EMGs.…”