Deep Convolutional and LSTM Networks on Multi-Channel Time Series Data for Gait Phase Recognition

Kreuzer, David; Münz, Michael

doi:10.3390/s21030789

Cited by 23 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, ref. [45] used a CNN-LSTM model to detect different phases of the gait process, and it achieved an accuracy of 92%, which is lower than this study's best score. Our results are comparable to those of [30] except for the cases where the model is reprocessed on personal historical data, making the model more adjustable to each participant.…”

Section: Comparison With the State-of-the-artcontrasting

confidence: 60%

See 1 more Smart Citation

Foot-to-Ground Phases Detection: A Comparison of Data Representation Formatting Methods with Respect to Adaption of Deep Learning Architectures

et al. 2022

View full text Add to dashboard Cite

Identifying the foot stance and foot swing phases, also known as foot-to-ground (FTG) detection, is a branch of Human Activity Recognition (HAR). Our study aims to detect two main phases of the gait (i.e., foot-off and foot-contact) corresponding to the moments when each foot is in contact with the ground or not. This will allow the medical professionals to characterize and identify the different phases of the human gait and their respective patterns. This detection process is paramount for extracting gait features (e.g., step width, stride width, gait speed, cadence, etc.) used by medical experts to highlight gait anomalies, stance issues, or any other walking irregularities. It will be used to assist health practitioners with patient monitoring, in addition to developing a full pipeline for FTG detection that would help compute gait indicators. In this paper, a comparison of different training configurations, including model architectures, data formatting, and pre-processing, was conducted to select the parameters leading to the highest detection accuracy. This binary classification provides a label for each timestamp informing whether the foot is in contact with the ground or not. Models such as CNN, LSTM, and ConvLSTM were the best fits for this study. Yet, we did not exclude DNNs and Machine Learning models, such as Random Forest and XGBoost from our work in order to have a wide range of possible comparisons. As a result of our experiments, which included 27 senior participants who had a stroke in the past wearing IMU sensors on their ankles, the ConvLSTM model achieved a high accuracy of 97.01% for raw windowed data with a size of 3 frames per window, and each window was formatted to have two superimposed channels (accelerometer and gyroscope channels). The model was trained to have the best detection without any knowledge of the participants’ personal information including age, gender, health condition, the type of activity, or the used foot. In other words, the model’s input data only originated from IMU sensors. Overall, in terms of FTG detection, the combination of the ConvLSTM model and the data representation had an important impact in outperforming other start-of-the-art configurations; in addition, the compromise between the model’s complexity and its accuracy is a major asset for deploying this model and developing real-time solutions.

show abstract

Section: Comparison With the State-of-the-artcontrasting

confidence: 60%

“…Ref. [45] also used this data representation in order to classify gait phases using a hybrid model composed of a CNN sub-model followed by a couple of LSTM layers. The ReLU (Rectified Linear Unit) activation function was mainly used for the different tested architectures.…”

Section: Concatenated Datamentioning

confidence: 99%

Foot-to-Ground Phases Detection: A Comparison of Data Representation Formatting Methods with Respect to Adaption of Deep Learning Architectures

et al. 2022

View full text Add to dashboard Cite

show abstract

“…It is capable of learning the characteristics of complex data faster than other RNN models [17]. Further, this technique has been employed for gait analysis and has become popular among scientists in recent studies [18][19][20].…”

Section: Feature Labelingmentioning

confidence: 99%

Gait Phase Estimation by Using LSTM in IMU-Based Gait Analysis—Proof of Concept

Sarshar

Polturi

Schega

2021

Sensors

View full text Add to dashboard Cite

Gait phase detection in IMU-based gait analysis has some limitations due to walking style variations and physical impairments of individuals. Therefore, available algorithms may not work properly when the gait data is noisy, or the person rarely reaches a steady state of walking. The aim of this work was to employ Artificial Intelligence (AI), specifically a long short-term memory (LSTM) algorithm, to overcome these weaknesses. Three supervised LSTM-based models were designed to estimate the expected gait phases, including foot-off (FO), mid-swing (MidS) and foot-contact (FC). For collecting gait data two tri-axial inertial sensors were located above each ankle. The angular velocity magnitude, rotation matrix magnitude and free acceleration magnitude were captured for data labeling and turning detection and to strengthen the model, respectively. To do so, a train dataset based on a novel movement protocol was acquired. A validation dataset similar to a train dataset was generated as well. Five test datasets from already existing data were also created to independently evaluate the models. After testing the models on validation and test datasets, all three models demonstrated promising performance in estimating desired gait phases. The proposed approach proves the possibility of employing AI-based algorithms to predict labeled gait phases from a time series of gait data.

show abstract

“…The same phenomenon can be expected in many practical applications where only one gait cycle worth of kinetic data is available. We wished to implement the CRNN architecture as it has recently been applied with success in related tasks such as video based person re-ID and gait phase recognition [45,46]. The input to each CRNN is a length normalized sequence of shape s b × C in × 400, where s b is batch size.…”

Section: Network Architecturementioning

confidence: 99%

The Most Discriminant Components of Force Platform Data for Gait Based Person Re-identification

Duncanson¹,

Thwaites²,

Booth³

et al. 2021

Preprint

View full text Add to dashboard Cite

Walking gait data measured using force platforms is a promising means for person re-identification in authentication and surveillance scenarios. We aimed to determine the most discriminant components of force platform data using a two-stream Convolutional Recurrent Neural Network (KineticNet). Each network in the two-stream architecture extracts features pertaining to a single stance phase and then these features are fused to represent the entire gait cycle. Over two sessions, ground reaction forces (Fx, Fy, Fz), moments (Mx, My, Mz), and center of pressure coordinates (Cx, Cy) were acquired from 118 participants as they walked our laboratory five times at preferred speed. For each participant and each session, up to three samples were reserved for network training, leaving one sample for network validation and one sample for network testing. KineticNet’s performance was evaluated using both individual component and multi-component inputs before ablation studies were conducted on its architecture. Fz was the most discriminant individual component, and re-identification using Fz, Fy, and Cy together was the most accurate overall at 96.02%. These results warrant further investigation into the utility of force platforms as an accessory or alternative to video cameras for gait based person re-identification.

show abstract

Deep Convolutional and LSTM Networks on Multi-Channel Time Series Data for Gait Phase Recognition

Cited by 23 publications

References 19 publications

Foot-to-Ground Phases Detection: A Comparison of Data Representation Formatting Methods with Respect to Adaption of Deep Learning Architectures

Foot-to-Ground Phases Detection: A Comparison of Data Representation Formatting Methods with Respect to Adaption of Deep Learning Architectures

Gait Phase Estimation by Using LSTM in IMU-Based Gait Analysis—Proof of Concept

The Most Discriminant Components of Force Platform Data for Gait Based Person Re-identification

Contact Info

Product

Resources

About