Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network

Abstract

Wearable inertial measurement unit (IMU) sensors are powerful enablers for acquisition of motion data. Specifically, in human activity recognition (HAR), IMU sensor data collected from human motion are categorically combined to formulate datasets that can be used for learning human activities. However, successful learning of human activities from motion data involves the design and use of proper feature representations of IMU sensor data and suitable classifiers. Furthermore, the scarcity of labelled data is an impeding factor in the process of understanding the performance capabilities of data-driven learning models. To tackle these challenges, two primary contributions are in this article: first; by using raw IMU sensor data, a spectrogram-based feature extraction approach is proposed. Second, an ensemble of data augmentations in feature space is proposed to take care of the data scarcity problem. Performance tests were conducted on a deep long term short term memory (LSTM) neural network architecture to explore the influence of feature representations and the augmentations on activity recognition accuracy. The proposed feature extraction approach combined with the data augmentation ensemble produces state-of-the-art accuracy results in HAR. A performance evaluation of each augmentation approach is performed to show the influence on classification accuracy. Finally, in addition to using our own dataset, the proposed data augmentation technique is evaluated against the University of California, Irvine (UCI) public online HAR dataset and yields state-of-the-art accuracy results at various learning rates.

The Use of Inertial Measurement Units for the Study of Free Living Environment Activity Assessment: A Literature Review

“…In particular, the use of recurring neural networks such as LSTM could be promising as they allow to remove the feature extraction step and directly work from raw signals. These techniques have successfully been used for activity classification in controlled environments [ 123 ]. The implementation of LSTM networks in FLEs or semi-FLEs could therefore represent a future possibility to improve the performances The use of IMUs sensors coupled with data analysis with Deep Learning models is becoming more and more frequent [ 124 , 125 ] and allows a more accurate analysis.…”

Gait Activity Classification on Unbalanced Data from Inertial Sensors Using Shallow and Deep Learning

“…We applied five techniques to generate synthetic data in order to balance the classes of the dataset used in the present study. These techniques that are commonly used in the field of activity recognition using time-series were analyzed [ 28 , 29 ], implementing those that best fit the data types. The selected techniques were grouped into: (i) Data Modifying the Magnitude of the signal’s (DMM), and (ii) Data Modifying the Frequency of the signals (DMF), and are described below: Scaling (DMM): Multiply all elements of each signal by a random value.…”

Acceleration Magnitude at Impact Following Loss of Balance Can Be Estimated Using Deep Learning Model

“…where ⊗, σ(x), W αβ , and b β were product, sigmoid function, weight matrix between α and β, and bias of β with β ∈ {i,f,c,o}, respectively [43]. A deep learning LSTM model allows different sizes of input vector for training and testing processes, unlike other classification algorithms such as a neural network.…”