Deep Sensing: Inertial and Ambient Sensing for Activity Context Recognition Using Deep Convolutional Neural Networks

Otebolaku, Abayomi Moradeyo; Enamamu, Timibloudi; Alfoudi, Ali Saeed Dayem; Ikpehai, Augustine; Marchang, Jims; Lee, Gyu Myoung

doi:10.3390/s20133803

Cited by 9 publications

(9 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We tuned the L2 weight regularisation for both the CNN layers and the LSTM cells. We confirmed from literature that the CNN layers usually require small L2 weight decay to perform optimally [59,63]. Dropout regularisation is another way to prevent the model from overfitting.…”

Section: Hyper-parameter Tuningsupporting

confidence: 71%

“…The LSTM Networks: an interesting part of the model in the introduction of the LSTM networks immediately after the Flatten layer. This is where the current work differs significantly to that conducted recently by researcher in Otebolaku et al in [59]. According to [22] for models that need to learn the temporal patterns in input data, preceding an LSTM network with a CNN sub-model always outperforms models with only one of the two.…”

Section: The Proposed Cnn-lstm Hybrid Modelmentioning

confidence: 63%

“…The F-score remains the most commonly used metric for class imbalanced datasets (He & Ma, 2013, p. 27). Considering the imbalance nature of the dataset and the tendency of the classifier to skew towards the more frequent classes during training, we adopted the macro-average Fscore previously used successfully by other scholars [47,59]. This metric truly reflects the representation of small classes within the dataset and it is calculated as follows:…”

Section: R = Tp/(tp+fn)mentioning

confidence: 99%

See 2 more Smart Citations

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Otebolaku¹,

Omolaja²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Smart devices such as smartphones, smartwatches, etc. are promising platforms that are being used for automatic recognition of human activities. However, it is difficult to accurately monitor complex human activities due to inter-class pattern similarity, which occurs when different human activities exhibit similar signal patterns or characteristics. Current smartphone-based recognition systems depend on the traditional sensors such as accelerometer and gyroscope, which are inbuilt in these devices. Therefore, apart from using information from the traditional sensors, these systems lack contextual information to support automatic activity recognition. In this article, we explore environment contexts such as illumination(light conditions) and noise level to support sensory data obtained from the traditional sensors using a hybrid of Convolutional Neural Networks and Long Short Time Memory(CNN_LSTM) learning models. The models performed sensor fusion by augmenting the low-level sensor signals with rich contextual data to improve recognition and generalisation ability of the proposed solution. Two sets of experiments were performed to validate the proposed solution. The first set of experiments used inertial sensing data whilst the second set of extensive experiments combined inertial signals with contextual information from environment sensing data. Obtained results demonstrate that contextual information such as environment noise level and illumination using hybrid deep learning models achieved better recognition accuracy than the traditional activity recognition models without contextual information.

show abstract

Section: Hyper-parameter Tuningsupporting

confidence: 71%

Section: The Proposed Cnn-lstm Hybrid Modelmentioning

confidence: 63%

Section: R = Tp/(tp+fn)mentioning

confidence: 99%

See 1 more Smart Citation

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Otebolaku¹,

Omolaja²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…More specifically, the proposed solution only uses sensor data from smartphones as opposed to other models that require multiple wearable and nonwearable sensors, such as video cameras, which are far from ideal and quite intrusive. Although there are similar works that demonstrate the adequacy of smartphone sensor data in activity recognition [53][54][55][56][60][61][62][63][64][65], to the best of our knowledge, the proposed work is the first to combine rich contextual information from environmental sensors with low-level sensor signals from the inertial sensors to distinguish between simple and complex activities and address associated interclass similarity problems using the combination of temporal features from both sliding window and LSTM.…”

Section: Machine Learning For Complex Harmentioning

confidence: 99%

Context-Aware Complex Human Activity Recognition Using Hybrid Deep Learning Models

2022

Self Cite

View full text Add to dashboard Cite

Smart devices, such as smartphones, smartwatches, etc., are examples of promising platforms for automatic recognition of human activities. However, it is difficult to accurately monitor complex human activities on these platforms due to interclass pattern similarities, which occur when different human activities exhibit similar signal patterns or characteristics. Current smartphone-based recognition systems depend on traditional sensors, such as accelerometers and gyroscopes, which are built-in in these devices. Therefore, apart from using information from the traditional sensors, these systems lack the contextual information to support automatic activity recognition. In this article, we explore environmental contexts, such as illumination (light conditions) and noise level, to support sensory data obtained from the traditional sensors using a hybrid of Convolutional Neural Network and Long Short-Term Memory (CNN–LSTM) learning models. The models performed sensor fusion by augmenting low-level sensor signals with rich contextual data to improve the models’ recognition accuracy and generalization. Two sets of experiments were performed to validate the proposed solution. The first set of experiments used triaxial inertial sensing signals to train baseline models, while the second set of experiments combined the inertial signals with contextual information from environmental sensors. The obtained results demonstrate that contextual information, such as environmental noise level and light conditions using hybrid deep learning models, achieved better recognition accuracy than the traditional baseline activity recognition models without contextual information.

show abstract

“…In order to alleviate the sample imbalance problem, some methods can play an important role in it. Research has investigated this problem by improving the performance of the recognition model [ 19 ]. In addition to modifying model, it can also be alleviated from the perspective of data processing.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders

Fan

Zhang

et al. 2020

Sensors

View full text Add to dashboard Cite

Activity recognition has received considerable attention in many research fields, such as industrial and healthcare fields. However, many researches about activity recognition have focused on static activities and dynamic activities in current literature, while, the transitional activities, such as stand-to-sit and sit-to-stand, are more difficult to recognize than both of them. Consider that it may be important in real applications. Thus, a novel framework is proposed in this paper to recognize static activities, dynamic activities, and transitional activities by utilizing stacked denoising autoencoders (SDAE), which is able to extract features automatically as a deep learning model rather than utilize manual features extracted by conventional machine learning methods. Moreover, the resampling technique (random oversampling) is used to improve problem of unbalanced samples due to relatively short duration characteristic of transitional activity. The experiment protocol is designed to collect twelve daily activities (three types) by using wearable sensors from 10 adults in smart lab of Ulster University, the experiment results show the significant performance on transitional activity recognition and achieve the overall accuracy of 94.88% on three types of activities. The results obtained by comparing with other methods and performances on other three public datasets verify the feasibility and priority of our framework. This paper also explores the effect of multiple sensors (accelerometer and gyroscope) to determine the optimal combination for activity recognition.

show abstract

Deep Sensing: Inertial and Ambient Sensing for Activity Context Recognition Using Deep Convolutional Neural Networks

Cited by 9 publications

References 35 publications

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Context-Aware Complex Human Activity Recognition using Hybrid Deep Learning Models

Context-Aware Complex Human Activity Recognition Using Hybrid Deep Learning Models

Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders

Contact Info

Product

Resources

About