Deep ConvLSTM Network with Dataset Resampling for Upper Body Activity Recognition Using Minimal Number of IMU Sensors

Abstract

Human activity recognition (HAR) is the study of the identification of specific human movement and action based on images, accelerometer data and inertia measurement unit (IMU) sensors. In the sensor based HAR application, most of the researchers used many IMU sensors to get an accurate HAR classification. The use of many IMU sensors not only limits the deployment phase but also increase the difficulty and discomfort for users. As reported in the literature, the original model used 19 sensor data consisting of accelerometers and IMU sensors. The imbalanced class distribution is another challenge to the recognition of human activity in real-life. This is a real-life scenario, and the classifier may predict some of the imbalanced classes with very high accuracy. When a model is trained using an imbalanced dataset, it can degrade model’s performance. In this paper, two approaches, namely resampling and multiclass focal loss, were used to address the imbalanced dataset. The resampling method was used to reconstruct the imbalanced class distribution of the IMU sensor dataset prior to model development and learning using the cross-entropy loss function. A deep ConvLSTM network with a minimal number of IMU sensor data was used to develop the upper-body HAR model. On the other hand, the multiclass focal loss function was used in the HAR model and classified minority classes without the need to resample the imbalanced dataset. Based on the experiments results, the developed HAR model using a cross-entropy loss function and reconstructed dataset achieved a good performance of 0.91 in the model accuracy and F1-score. The HAR model with a multiclass focal loss function and imbalanced dataset has a slightly lower model accuracy and F1-score in both 1% difference from the resampling method. In conclusion, the upper body HAR model using a minimal number of IMU sensors and proper handling of imbalanced class distribution by the resampling method is useful for the assessment of home-based rehabilitation involving activities of daily living.

Upper Limbs Extension and Flexion Angles Calculation and Visualization Using Two Wearable Inertial Measurement Units

“…Increasing number of IMU sensors will increase the cost and burden to the individual who want to use it at home. In [13], resampling the dataset and multiclass focal loss technique were used to address the imbalanced dataset and reduced the usage of IMU sensors with comparable results with previous works. Beside IMU sensors, kinematics using video processing methods has been study for home-based stroke rehabilitation program [15].…”

“…Multiple IMU sensors can be used classified human activities using deep ConvLSTM network [13][14]. Increasing number of IMU sensors will increase the cost and burden to the individual who want to use it at home.…”

ECG-NETS – A novel integration of capsule networks and extreme gated recurrent neural network for IoT based human activity recognition

“…model in detecting HAR under different datasets. CNN+GRU [35], Deep CONVLSTM [36] networks delivered the better order exhibitions. In these two cases, LSTM [38] and inception model [37] the grouping exhibitions than existing hybrid learning calculations.…”

Task grouping and optimized deep learning based VM sizing for hosting containers as a service

“…The task groups thus formed are denoted by G1, G2, and G3. After task grouping, VM sizing is established by using the Deep Convolutional LSTM Network (Deep-ConvLSTM) [27,28] based on the task parameters of the groups. The Deep-ConvLSTM is tuned with the introduced Fractional Pelican Optimization (FPO) Algorithm.…”