ConvAE-LSTM: Convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition

Thakur, Dipanwita; Biswas, Suparna; Ho, Edmond S. L.; Chattopadhyay, Samiran

doi:10.1109/access.2022.3140373

Cited by 49 publications

(29 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This further demonstrates the dominance of deep learning models over classical algorithms that rely heavily on heuristic hand-crafted features [11]. Interestingly, it also justified the applicability of the CNN-LSTM hybrid model to achieve state-of-the-art results with time series data, more importantly with activity context recognition [14,51,53,61,70] and speech recognition problems [22].…”

Section: Discussionmentioning

confidence: 72%

See 1 more Smart Citation

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Otebolaku¹,

Omolaja²

2022

Preprint

View full text Add to dashboard Cite

Smart devices such as smartphones, smartwatches, etc. are promising platforms that are being used for automatic recognition of human activities. However, it is difficult to accurately monitor complex human activities due to inter-class pattern similarity, which occurs when different human activities exhibit similar signal patterns or characteristics. Current smartphone-based recognition systems depend on the traditional sensors such as accelerometer and gyroscope, which are inbuilt in these devices. Therefore, apart from using information from the traditional sensors, these systems lack contextual information to support automatic activity recognition. In this article, we explore environment contexts such as illumination(light conditions) and noise level to support sensory data obtained from the traditional sensors using a hybrid of Convolutional Neural Networks and Long Short Time Memory(CNN_LSTM) learning models. The models performed sensor fusion by augmenting the low-level sensor signals with rich contextual data to improve recognition and generalisation ability of the proposed solution. Two sets of experiments were performed to validate the proposed solution. The first set of experiments used inertial sensing data whilst the second set of extensive experiments combined inertial signals with contextual information from environment sensing data. Obtained results demonstrate that contextual information such as environment noise level and illumination using hybrid deep learning models achieved better recognition accuracy than the traditional activity recognition models without contextual information.

show abstract

Section: Discussionmentioning

confidence: 72%

“…For both of the experiments, we used the following metrics: Precision, Recall, F-Score and the Confusion matrix. According to many authors, these are the most commonly used metrics in the field of activity recognition [10,[69][70]. As clearly illustrated in Figure 10, the dataset suffers from class imbalance.…”

Section: Evaluation Metricsmentioning

confidence: 99%

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Otebolaku¹,

Omolaja²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Additionally, CNN and LSTM are widely adopted with high accuracy rate activity recognition among the applied networks. CNN is commonly separated into numerous learning stages, each of which consists of a mix of convolutional operation and nonlinear processing units, as follows:

where

reveals the latent representation of the

-th feature map of the current layer,

is the activation function,

denotes the convolution operation,

indicates the

-th feature map of the group of the feature maps

achieved from the upper layer,

and

express the weights matrix and the bias of the

-th feature map of the current layer, receptively [ 52 ]. In our model, the rectified linear units (ReLU) were employed as the activation functions to subsequently conduct the non-linear transformation to obtain the feature maps, denoted by:

…”

Section: Methodsmentioning

confidence: 99%

“…CNN considers each frame of sensor data as independent and extracts the features for these isolated portions of data without considering the temporal contexts beyond the boundaries of the frame. Due to the continuity of sensor data flow produced by the user’s behavior, local spatial correlations and temporally long-term connections are both important to identify the landmark [ 52 ]. LSTMs with learnable gates, which modulate the flow of information and control when to forget previous hidden states, as variants of vanilla recurrent neural networks (RNNs), allow the neural network to effectively extract the long-range dependencies of time-series sensor data [ 54 ].…”

Section: Methodsmentioning

confidence: 99%

Indoor Spatiotemporal Contact Analytics Using Landmark-Aided Pedestrian Dead Reckoning on Smartphones

Gao

Konomi

2022

Sensors

View full text Add to dashboard Cite

Due to the prevalence of COVID-19, providing safe environments and reducing the risks of virus exposure play pivotal roles in our daily lives. Contact tracing is a well-established and widely-used approach to track and suppress the spread of viruses. Most digital contact tracing systems can detect direct face-to-face contact based on estimated proximity, without quantifying the exposed virus concentration. In particular, they rarely allow for quantitative analysis of indirect environmental exposure due to virus survival time in the air and constant airborne transmission. In this work, we propose an indoor spatiotemporal contact awareness framework (iSTCA), which explicitly considers the self-containing quantitative contact analytics approach with spatiotemporal information to provide accurate awareness of the virus quanta concentration in different origins at various times. Smartphone-based pedestrian dead reckoning (PDR) is employed to precisely detect the locations and trajectories for distance estimation and time assessment without the need to deploy extra infrastructure. The PDR technique we employ calibrates the accumulative error by identifying spatial landmarks automatically. We utilized a custom deep learning model composed of bidirectional long short-term memory (Bi-LSTM) and multi-head convolutional neural networks (CNNs) for extracting the local correlation and long-term dependency to recognize landmarks. By considering the spatial distance and time difference in an integrated manner, we can quantify the virus quanta concentration of the entire indoor environment at any time with all contributed virus particles. We conducted an extensive experiment based on practical scenarios to evaluate the performance of the proposed system, showing that the average positioning error is reduced to less than 0.7 with high confidence and demonstrating the validity of our system for the virus quanta concentration quantification involving virus movement in a complex indoor environment.

show abstract

“…The proposed model was a sequential fusion of CNNs, LSTMs, and fully connected layers. Thakur et al 21 proposed a DL-based unified model composed of CNNs, autoencoders, and LSTMs. The model learnt both spatial features and temporal features from smartphone sensor data.…”

Section: Introductionmentioning

confidence: 99%

A Novel CNN-based Bi-LSTM parallel model with attention mechanism for human activity recognition with noisy data

Yin

Liu

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Boosted by mobile communication technologies, Human Activity Recognition (HAR) based on smartphones has attracted more and more attentions of researchers. One of the main challenges is the classification time and accuracy in processing long-time dependent sequence samples with noisy or missed data. In this paper, a 1-D Convolution Neural Network (CNN)-based bi-directional Long Short-Term Memory (LSTM) parallel model with attention mechanism (ConvBLSTM-PMwA) is proposed. The original features of sensors are segmented into sub-segments by well-designed equal time step sliding window, and fed into 1-D CNN-based bi-directional LSTM parallel layer to accelerate feature extraction with noisy and missed data. The weights of extracted features are redistributed by attention mechanism and integrated into complete features. At last, the final classification results are obtained with the full connection layer. The performance is evaluated on public UCI and WISDM HAR datasets. The results show that the ConvBLSTM-PMwA model performs better than the existing CNN and RNN models in both classification accuracy (96.71%) and computational time complexity (1.1 times faster at least), even if facing HAR data with noise.

show abstract

ConvAE-LSTM: Convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition

Cited by 49 publications

References 95 publications

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Indoor Spatiotemporal Contact Analytics Using Landmark-Aided Pedestrian Dead Reckoning on Smartphones

A Novel CNN-based Bi-LSTM parallel model with attention mechanism for human activity recognition with noisy data

Contact Info

Product

Resources

About