Gait Activity Classification Using Multi-Modality Sensor Fusion: A Deep Learning Approach

Abstract

Floor Sensors (FS) are used to capture information from the force induced on the contact surface by feet during gait. On the other hand, the Ambulatory Inertial Sensors (AIS) are used to capture the velocity, acceleration and orientation of the body during different activities. In this paper, fusion of the stated modalities is performed to overcome the challenge of gait classification from wearable sensors on the lower portion of human body not in contact with ground as in FS. Deep learning models are utilized for the automatic feature extraction of the ground reaction force obtained from a set of 116 FS and body movements from AIS attached at 3 different locations of lower body, which is novel. Spatio-temporal information of disproportionate inputs obtained from the two modalities is balanced and fused within deep learning network layers whilst reserving the categorical content for each gait activity. Our approach of fusion compensates the degradation in spatio-temporal accuracies in individual modalities and makes the overall classification outcomes more accurate. Further assessment of multi-modality based results show significant improvements in f-scores using different deep learning models i.e., LSTM (99.90%), 2D-CNN (88.73%), 1D-CNN (94.97%) and ANN (89.33%) respectively.

Computer Vision System Based on the Analysis of Gait Features for Fall Risk Assessment in Elderly People

“…These points are then exported to Python, where they are used to train a DL model. For the development of the algorithm, the use of the open-source library Keras in Python was chosen, which is designed for the training and implementation of DL models [36,37]. These models can be used both for the development of neural networks for the analysis of statistical databases and for the training of models for the analysis of images, which perform tasks of detection of objects, people, faces, and body orientation, to name a few.…”

Gait Recognition for Lower Limb Exoskeletons Based on Interactive Information Fusion

“…As shown in Figure 5 , the typical five stages are the prestance, midstance, terminal stance, preswing, and terminal swing, respectively. In recent study, Yan et al [ 120 ] divided the gait cycle into 4 stages and proposed a new voting weighting method to integrate the multidimensional acceleration signals collected by inertial measurement unit (IMU) into the voting-weighted integrated neural network (VWI-DNN) algorithm model and its classification accuracy, and Macro-F1 is up to 99.5%.…”

Validation of Step Detection and Distance Calculation Algorithms for Soccer Performance Monitoring

“…Architectures like Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs) have proven adept at extracting complex patterns from IMU-generated time-series data [ 34 , 35 ]. This has led to their application in step detection [ 36 , 37 , 38 , 39 , 40 ] and stride length estimation [ 41 ]. Despite these advancements, deep learning models pose challenges, especially in real-time and resource-constrained environments typical of wearable devices.…”