Multi-modality fusion of floor and ambulatory sensors for gait classification

Yunas, Syed U.; Alharthi, Abdullah; Ozanyan, Krikor B.

doi:10.1109/isie.2019.8781127

Cited by 7 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gait events, such as initial contact and toe off, were detected with an accuracy of 100% using IMUs and force sensors installed in shoes in [54]. A kernel SVM was fed features extracted from IMUs and plastic optical fiber-based floor sensors by using PCA and canonical correlation analysis to identify humans with an accuracy of 94% in [31]. Kinetic and kinematic gait features obtained from a 3D motion analysis system and two force plates were fed to linear discriminant analysis and quadratic discriminant analysis classifiers to classify ASD with an accuracy of 82.50% in [22].…”

Section: Related Workmentioning

confidence: 99%

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Jun

Lee

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Classification of pathological gaits has an important role in finding a weakened body part or disease and supporting a doctor's decision. Many machine learning-based approaches have been proposed that automatically classify abnormal gait patterns using various sensors, such as inertial sensors, depth cameras and foot pressure plates. In this paper, we present a deep learning-based abnormal gait classification method employing both a 3D skeleton (obtained with a depth camera) and plantar foot pressure. We collected skeleton and foot pressure data simultaneously for 1 normal and 5 pathological gaits (antalgic, lurching, steppage, stiff-legged, and Trendelenburg gaits) and classified the gaits by using single-modal models fed either skeleton or pressure data and a multimodal model fed both data types together. In the proposed method, we fed the sequential skeleton and average foot pressure data into recurrent neural network (RNN)-based encoding layers and convolutional neural network (CNN)-based encoding layers, respectively. Finally, the output features were concatenated and fed to the classification layers. The pressure-based and skeleton-based single-modal models achieved classification accuracies of 68.82% and 93.40%, respectively. The proposed multimodal hybrid model using skeleton and foot pressure data together showed improved performance, with an accuracy of 95.66%. We fine-tuned the hybrid model by applying a 3-step training methodology and ultimately increased the accuracy to 97.60%. This study indicates that the integrated features of the skeleton and foot pressure data represent both the spatiotemporal motion information and weight distribution, so data fusion can generate a positive effect in pathological gait classification.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Jun

Lee

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Therefore, in this article, the main evaluation metrics will be presented along with their formula, and those who are interested can refer to the relevant paper to get acquainted with other metrics. You need to know this as a recommendation from us that accuracy in this field is introduced either based on the formula [correct/total] [94], [109] or based on [(tp+tn)/(tp+tn+fp+fn)] [156], [157], [158], [159], [160], [161], [162]; this, metric alone, especially with imbalanced data, is not a good measure of classification performance. Therefore, the authors of this article strongly recommend that other metrics are be used to report classification results too.…”

Section: J Evaluationmentioning

confidence: 99%

Statistical Database of Human Motion Recognition Using Wearable IoT—A Review

et al. 2023

View full text Add to dashboard Cite

Wearable sensors and the Internet of Things (IoT) will be two buzzwords that will be heard commonly in the coming decades. The combination of these two technologies soon will create a great revolution in applications that require motion recognition, such as health care, sports, and entertainment. The development of technology has made wearable sensors one of the most basic tools for human motion analysis. We believe that IoT is the most powerful complement to the use of wearable sensors in the analysis of human body motion. Using wearable IoT, all necessary human data will be collected and delivered via the Internet to the experts who can make accurate decisions about the type of activity, falling situations, freezing of gait (fog), and so on. In this article, the human motion analysis is presented in a chart and is divided into two parts: movement measurement and movement classification. However, this article focuses on movement classification that includes three subsections, gait analysis (GA), gesture recognition (GR), and human activity recognition (HAR), and is closely related to human motion recognition. In this article, our goal is to first acquaint the reader with the important steps required to classify the movement of the human body by wearable sensors and then by using tables to determine the most used algorithms and methods for each step. After briefly reviewing IoT concepts, directions for further research will be provided.

show abstract

“…The architecture of the CNN model engineered for this study is shown in figure 2. The model is proposed based on extensive research on gait in our previous work [14], [15], [16], [17]. The model consists of 12 stacked layers to .…”

Section: Convolutional Neural Network (Cnn)mentioning

confidence: 99%

“…Therefore, the goal of this work is twofold: firstly, to categorize gait parameters in healthy adults while performing cognitive demanding tasks, using automatic extraction of optimal gait features by deep CNN; secondly, to interpret the model performance on unseen spatiotemporal signals by applying the technique of Layer-Wise Relevance Propagation (LRP) [12] to attempt linking key known events in the gait cycle to cognitive deterioration. As a main data acquisition methodology, the subject's unique gait signatures based on GRF signals were recorded using a set of multiple floor sensors based on plastic optical fiber (POF) technology, in a system specially designed for optimal spatiotemporal sampling [13], [14]. The main sensor fusion and data processing methodology is deep CNNs with LRP applied on the classifications to allow interpretation of the behavioral changes due to a demanding task while walking.…”

Section: Introductionmentioning

confidence: 99%

Sensor Fusion for Analysis of Gait under Cognitive Load: Deep Learning Approach

Alharthi

Yunas

Ozanyan

2020

2020 IEEE Sensors Applications Symposium (SAS)

Self Cite

View full text Add to dashboard Cite

Human mobility requires substantial cognitive resources, thus elevated complexity in the navigated environment instigates gait deterioration due to naturally limited cognitive load capacity. This work uses deep learning methods for 116 sensors fusion to study the effects of cognitive load on human gait of healthy subjects. We demonstrate classifications, achieving 86% precision with Convolutional Neural Networks (CNN), of normal gait as well as 15 subjects' gait under two types of cognitive demanding tasks. Floor sensors capturing multiples of up to 4 uninterrupted steps were utilized to harvest the raw gait spatiotemporal signals, based on the ground reaction force (GRF). A Layer-Wise Relevance Propagation (LRP) technique is proposed to interpret the CNN prediction in terms of relevance to standard events in the gait cycle. LRP projects the model predictions back to the input gait spatiotemporal signal, to generate a "heat map" over the original training set, or an unknown sample classified by the model. This allows valuable insight into which parts of the gait spatiotemporal signal have the heaviest influence on the gait classification and consequently, which gate events are mostly affected by cognitive load.

show abstract

Multi-modality fusion of floor and ambulatory sensors for gait classification

Cited by 7 publications

References 18 publications

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Statistical Database of Human Motion Recognition Using Wearable IoT—A Review

Sensor Fusion for Analysis of Gait under Cognitive Load: Deep Learning Approach

Contact Info

Product

Resources

About