CNN-Based Estimation of Sagittal Plane Walking and Running Biomechanics From Measured and Simulated Inertial Sensor Data

Abstract

Machine learning is a promising approach to evaluate human movement based on wearable sensor data. A representative dataset for training data-driven models is crucial to ensure that the model generalizes well to unseen data. However, the acquisition of sufficient data is time-consuming and often infeasible. We present a method to create realistic inertial sensor data with corresponding biomechanical variables by 2D walking and running simulations. We augmented a measured inertial sensor dataset with simulated data for the training of convolutional neural networks to estimate sagittal plane joint angles, joint moments, and ground reaction forces (GRFs) of walking and running. When adding simulated data, the root mean square error (RMSE) of the test set of hip, knee, and ankle joint angles decreased up to 17 %, 27 % and 23 %, the RMSE of knee and ankle joint moments up to 6 % and the RMSE of anterior-posterior and vertical GRF up to 2 and 6 %. Simulation-aided estimation of joint moments and GRFs was limited by inaccuracies of the biomechanical model. Improving the physics-based model and domain adaptation learning may further increase the benefit of simulated data. Future work can exploit biomechanical simulations to connect different data sources in order to create representative datasets of human movement. In conclusion, machine learning can benefit from available domain knowledge on biomechanical simulations to supplement cumbersome data collections.

Predicting continuous ground reaction forces from accelerometers during uphill and downhill running: A recurrent neural network solution

“…Excluding foot strike pattern data affected the MAPE of discrete variables by < 3% (Table 3). These findings indicate that the LSTM network can predict normal GRF waveforms from a single accelerometer on the sacrum more accurately than neural networks implemented in previous studies (RMSE = 0.21 -0.39 BW, rRMSE = 13.92%), which require data from 3 -7 wearable devices (Wouda et al, 2018;Dorschky et al, 2020;Johnson et al, 2021). Step Frequency 0.1 ± 0.1% 0.1 ± 0.1% Contact Time 4.9 ± 4.0% 5.6 ± 4.5% Impulse 6.4 ± 6.9% 6.0 ± 7.1% Active Peak 8.5 ± 8.2% 7.7 ± 6.3% Loading Rate 27.6 ± 36.1% 30.3 ± 41.6% Table 3.…”

“…Our findings indicate that an LSTM network given the runner's mass, height, running speed, slope, foot strike pattern, and sacral acceleration can predict the normal GRF waveform across a range of speeds and slopes with an RMSE of 0.12 -0.20 BW and rRMSE of 5.4 -7.3% (Figure 4). For comparison, recent studies report an RMSE of 0.39 ± 0.26 BW (Wouda et al, 2018), an RMSE of 0.21 ± 0.03 BW (Dorschky et al, 2020), and an rRMSE of 13.92% (Johnson et al, 2021) when using neural networks to predict the stance phase vertical GRF waveform during level-ground running. Previous studies also quantified the accuracy of their networks using the Pearson correlation between the predicted and measured GRF waveforms, but we chose not to include correlation as a primary metric of accuracy.…”

“…However, IMUs cannot directly measure GRFs and require the use of algorithms to estimate discrete variables like peak vertical GRF, ground contact time, vertical impulse, and vertical loading rate (Neugebauer, Hawkins and Beckett, 2012;Ancillao et al, 2018;Kiernan et al, 2018;Derie et al, 2020;Alcantara et al, 2021). Recently, neural networks have been used to predict GRF waveforms during running (Wouda et al, 2018;Dorschky et al, 2020;Pogson et al, 2020;Johnson et al, 2021), from which a variety of discrete variables can be calculated. Although predictions of the entire GRF waveform represent a more versatile outcome compared to predicting a discrete variable, previous studies have used neural network architectures that required data to be time-normalized to the duration of a step (Dorschky et al, 2020) or stance phase (Wouda et al, 2018;Johnson et al, 2021), preventing the calculation of biomechanical variables with a non-normalized temporal component (e.g.…”

“…Pogson et al (2020) zero-padded the end of accelerometer data to the length of the longest stance phase to preserve the temporal component, but observed decreased accuracy when predicting the vertical GRF waveform during the stance phase (Root Mean Square Error ≈ 200 N) (Pogson et al, 2020). Additionally, previous studies have only predicted GRF waveforms during level-ground running (Wouda et al, 2018;Dorschky et al, 2020;Pogson et al, 2020;Johnson et al, 2021), limiting the application to environments where no uphill or downhill running occurs (i.e. a level treadmill or athletics track).…”

“…The purpose of this exploratory study was to develop an LSTM network that could predict the normal (perpendicular to running surface) GRF waveform across a range of running speeds and slopes using data from accelerometers. Given the novelty of this approach, we sought to develop a network that could predict the GRF waveform with accuracy better than state-of-the-art predictions of time-normalized vertical GRF data during level ground running using data from multiple IMUs: a Root Mean Square Error (RMSE) of 0.21 BW (Dorschky et al, 2020) and relative RMSE (rRMSE; RMSE normalized to the average range of the compared waveforms; Eq. 1) of 13.92% (Johnson et al, 2021).…”