Novel Deep Learning Network for Gait Recognition Using Multimodal Inertial Sensors

Abstract

Some recent studies use a convolutional neural network (CNN) or long short-term memory (LSTM) to extract gait features, but the methods based on the CNN and LSTM have a high loss rate of time-series and spatial information, respectively. Since gait has obvious time-series characteristics, while CNN only collects waveform characteristics, and only uses CNN for gait recognition, this leads to a certain lack of time-series characteristics. LSTM can collect time-series characteristics, but LSTM results in performance degradation when processing long sequences. However, using CNN can compress the length of feature vectors. In this paper, a sequential convolution LSTM network for gait recognition using multimodal wearable inertial sensors is proposed, which is called SConvLSTM. Based on 1D-CNN and a bidirectional LSTM network, the method can automatically extract features from the raw acceleration and gyroscope signals without a manual feature design. 1D-CNN is first used to extract the high-dimensional features of the inertial sensor signals. While retaining the time-series features of the data, the dimension of the features is expanded, and the length of the feature vectors is compressed. Then, the bidirectional LSTM network is used to extract the time-series features of the data. The proposed method uses fixed-length data frames as the input and does not require gait cycle detection, which avoids the impact of cycle detection errors on the recognition accuracy. We performed experiments on three public benchmark datasets: UCI-HAR, HuGaDB, and WISDM. The results show that SConvLSTM performs better than most of those reporting the best performance methods, at present, on the three datasets.

Building Carbon Emissions Prediction Based on Deep Learning Network with Improved Particle Swarm Optimization

“…A strong nonlinear modeling tool is the DNN, an artificial neural network having several layers between the input and output layers [20]. Figure 2 depicts the DNN's fundamental structure.…”

Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion

“…The CNN-LSTM network is a hybrid neural network that has achieved a wide range of applications in several fields, including emotion recognition [29], video action classification [30], pasta product classification [31], facial micro-expression recognition [32], gait recognition [33], and stock prediction [34]. Its main framework is made of a convolutional layer spliced with a BiLSTM layer, and then the attention mechanism and other networks can be added according to the data characteristics and task requirements.…”

EFE-LSTM: A Feature Extension, Fusion and Extraction Approach Using Long Short-Term Memory for Navigation Aids State Recognition

“…Wang et al [20] further leveraged data from smartphone sensors to construct a neural network model, significantly improving the accuracy of identifying different traffic states. Shi et al [23] constructed a gait recognition LSTM network based on multimodal wearable inertial sensor data with features automatically extracted. Nevertheless, the methods proposed in these studies still face limitations in obtaining data, and the correlation with device states is not sufficiently prominent.…”