Gait Trajectory Prediction on an Embedded Microcontroller Using Deep Learning

Abstract

Achieving a normal gait trajectory for an amputee’s active prosthesis is challenging due to its kinematic complexity. Accordingly, lower limb gait trajectory kinematics and gait phase segmentation are essential parameters in controlling an active prosthesis. Recently, the most practiced algorithm in gait trajectory generation is the neural network. Deploying such a complex Artificial Neural Network (ANN) algorithm on an embedded system requires performing the calculations on an external computational device; however, this approach lacks mobility and reliability. In this paper, more simple and reliable ANNs are investigated to be deployed on a single low-cost Microcontroller (MC) and hence provide system mobility. Two neural network configurations were studied: Multi-Layered Perceptron (MLP) and Convolutional Neural Network (CNN); the models were trained on shank and foot IMU data. The data were collected from four subjects and tested on a fifth to predict the trajectory of 200 ms ahead. The prediction was made for two cases: with and without providing the current phase of the gait. Then, the models were deployed on a low-cost microcontroller (ESP32). It was found that with fewer data (excluding the current gait phase), CNN achieved a better correlation coefficient of 0.973 when compared to 0.945 for MLP; when including the current phase, both network configurations achieved better correlation coefficients of nearly 0.98. However, when comparing the execution time required for the prediction on the intended MC, MLP was much faster than CNN, with an execution time of 2.4 ms and 142 ms, respectively. In summary, it was found that when training data are scarce, CNN is more efficient within the acceptable execution time, while MLP achieves relative accuracy with low execution time with enough data.

Implementing Gait Kinematic Trajectory Forecasting Models on an Embedded System

“…While there are intricate complexities related to safety regulatory requirements, user acceptance, as well as device reliability and adaptability that need to be considered, assistive ambulatory devices have the potential to help in such situations and may even reduce the burden on healthcare resources [ 4 ]. Nevertheless, given the importance of mobility assistance for both medical and non-medical end-user applications, there has been increasing interest in designing more effective and intelligent assistive ambulatory technologies [ 1 , 4 , 5 , 6 , 8 , 9 , 11 , 12 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 ].…”

“…This technology aims to achieve natural, stable, and accurate interactive control with respect to human motion intention [ 3 ]. It can be used to reduce injury to users as well as compensate for delays in the response time of more complex control systems [ 1 , 3 , 17 , 18 , 22 , 23 , 30 ]. Forecasting gait kinematics can also enable targeted functional electrical stimulation at specific points in the gait cycle, leading to more effective rehabilitation therapy, and may even minimize the risk of falling by detecting deviations in the anticipated gait trajectory [ 1 , 23 , 28 , 29 ].…”

“…The advancement in sensor-aided gait kinematic forecasting methods that can incorporate ongoing user-specific sensor signal information on an embedded system could have a significant impact on the design of intelligent lower-limb assistive devices [ 1 ]. Unlike fixed-motion capture systems, embedded systems offer a low-cost approach for the collection of high-quality, multi-source data for motion control purposes [ 2 , 3 ].…”

“…For instance, the deployment of sensor-aided gait kinematic forecasting technologies on embedded systems can be used to help reduce damage to joints, enhance rehabilitation outcomes, and even reduce the healthcare economic burden by lowering total healthcare system utilization [ 4 , 5 ]. While embedded systems can support the use of gait kinematic forecasting models for a variety of integrated robotic technologies, there are important limitations related to computational load when using complex state-of-the-art models on embedded systems [ 1 ]. To address this, the primary objective of this study was to deploy and evaluate a new computationally lightweight prediction algorithm with reduced complexity and demand for substantial computational resources to enable more effective gait kinematic forecasting on an embedded system.…”

Interpretable long-term gait trajectory prediction based on Interpretable-Concatenation former

“…Karakish et al and Kang et al used a CNN-based deep neural network to predict gait [10, 11], which is faster than the LSTM model. However, the model proposed by Karakish et al can only predict the gait trajectory in a short time, and the model proposed by Kang et al relies on user-specific basis data.…”