Deep Learning for Human Activity Recognition Based on Causality Feature Extraction

Abstract

We propose a novel data-driven feature extraction approach based on direct causality and fuzzy temporal windows (FTWs) to improve the precision of human activity recognition and mitigate the problems of easily-confused activities and unlabeled data, which significantly degrade classification performance owing to the correlation of labeled data. In recognizing activities, the proposed approach not only considers the importance of oncoming short-term sensor data but also considers the continuity from past activities of the preceding long-term sensor data. In terms of the oncoming data, the causality feature is extracted using the direct transfer entropy to determine the unique pattern of an activity, which represents the quantified causal relationship between sensor activations. In terms of the preceding data, several hours of historical data are compressed to fuzzy features based on FTWs. Subsequently, the causality and fuzzy features are fused by matrix multiplication to express distinct features of activities. To effectively learn the spatiotemporal dependencies of the fused feature, deep long short-term memory (LSTM), two-dimensional convolutional neural network (2D-CNN), and hybrid models composed of a combination of LSTM and CNN were used. Leave-one-day-out cross-validation was performed based on the CASAS open datasets, including Aruba, Cairo, and Milan. The results showed that the macro-F1-scores were improved by 16.4, 37.5, and 18.5%, respectively, compared with those of the FTW-only environments. In addition, the proposed approach could improve the precision of activity recognition and mitigate the problems associated with the environments containing unlabeled data.INDEX TERMS activity of daily living, activity recognition, causality, convolutional neural network, deep learning, fuzzy feature, long short-term memory.

Passive Infrared Sensor-Based Occupancy Monitoring in Smart Buildings: A Review of Methodologies and Machine Learning Approaches

“…By incorporating external features such as previous activity and begin time-stamp, the model enhances accuracy in recognizing daily activities, demonstrating an F1 score of 0.917, which is a notable improvement over state-of-the-art models. In study by Hwang et al [ 105 ], a new model using deep learning was developed, emphasizing causality feature extraction and fuzzy temporal windows (FTWs) for better precision. The model, tested on Aruba, Cairo, and Milan datasets, effectively distinguishes between easily-confused activities and manages unlabeled data challenges.…”

Enhancing Human Activity Recognition in Smart Homes with Self-Supervised Learning and Self-Attention

“…Although the DCNN model effectively extracted features from binary activity images, it overlooked temporary dependencies between activities, such as "Wash dishes" mostly occurring after "Meal preparation". Hybrid models, comprising a combination of LSTM and CNN [29], were employed to address this limitation. These models considered the importance of upcoming short-term sensor data and the continuity from past activities in preceding long-term sensor data.…”

Modified Wild Horse Optimization with Deep Learning Enabled Symmetric Human Activity Recognition Model

“…The time sequence classifier tasks were the main difficulties in utilizing HAR, which is if individual movements were estimated using sensory information. This normally includes precisely engineering features from the basic information through signal processing methods and deep domain expertise for fitting one of the methods of machine learning (ML) [9]. Recently, deep learning (DL) approaches, which include LSTM and CNN, automatically derive useful features from the raw sensor information and get an advanced outcome [10].…”