Zero-Shot Human Activity Recognition Using Non-Visual Sensors

The necessity of caring for elderly people is increasing. Great efforts are being made to enable the elderly population to remain independent for as long as possible. Technologies are being developed to monitor the daily activities of a person to detect their state. Approaches that recognize activities from simple environment sensors have been shown to perform well. It is also important to know the habits of a resident to distinguish between common and uncommon behavior. In this paper, we propose a novel approach to discover a person’s common daily routines. The approach consists of sequence comparison and a clustering method to obtain partitions of daily routines. Such partitions are the basis to detect unusual sequences of activities in a person’s day. Two types of partitions are examined. The first partition type is based on daily activity vectors, and the second type is based on sensor data. We show that daily activity vectors are needed to obtain reasonable results. We also show that partitions obtained with generalized Hamming distance for sequence comparison are better than partitions obtained with the Levenshtein distance. Experiments are performed with two publicly available datasets.

Section: Related Workmentioning

confidence: 99%

Discovering Daily Activity Patterns from Sensor Data Sequences and Activity Sequences

Maučec

Donaj

2021

“…[ 17 ] has proposed a multi-modal generative adversarial network (M2GAN) to fuse various types of class semantic prototypes, which are achieved in an adversarial framework. Machot et al [ 21 ] have designed a ZSL algorithm by exploiting heterogeneous knowledge between sensor data and semantic space, and then they have spread this algorithm from recognizing unseen classes to unseen human action. Matsuki et al [ 22 ] have proposed an extended word vector-based algorithm by analyzing several ZSL results of embedding semantic features in semantic space.…”

Section: Related Workmentioning

confidence: 99%

Zero-Shot Image Classification Based on a Learnable Deep Metric

Liu

Shi

et al. 2021

The supervised model based on deep learning has made great achievements in the field of image classification after training with a large number of labeled samples. However, there are many categories without or only with a few labeled training samples in practice, and some categories even have no training samples at all. The proposed zero-shot learning greatly reduces the dependence on labeled training samples for image classification models. Nevertheless, there are limitations in learning the similarity of visual features and semantic features with a predefined fixed metric (e.g., as Euclidean distance), as well as the problem of semantic gap in the mapping process. To address these problems, a new zero-shot image classification method based on an end-to-end learnable deep metric is proposed in this paper. First, the common space embedding is adopted to map the visual features and semantic features into a common space. Second, an end-to-end learnable deep metric, that is, the relation network is utilized to learn the similarity of visual features and semantic features. Finally, the invisible images are classified, according to the similarity score. Extensive experiments are carried out on four datasets and the results indicate the effectiveness of the proposed method.

“…Xie et al [45] proposed a hybrid system that used an inertial sensor and a barometer to learn static activities and detect locomotion. The authors in [46] proposed an approach based on zero-shot learning to learn human activities. Hamad et al [47] proposed a fuzzy windowing approach to extract temporal features for human activity recognition.…”

Section: Related Workmentioning

confidence: 99%

“…The F-score is favored over accuracy when we have an imbalanced data set due to its inherent capacity to measure a recognition method performance on a multi-class classification problem, and because it is adapted for the class distributions of the ground truth and the classified activity labels. Additionally, it is commonly used as an evaluation measure in smart home settings for evaluating the performances of human activity recognition methods as in [46,63,64]. Therefore, we follow a similar approach by using the F-score measure.…”

Section: Evaluation Criteriamentioning

confidence: 99%

Discovering Human Activities from Binary Data in Smart Homes

Eldib

Philips

Aghajan

2020

With the rapid development in sensing technology, data mining, and machine learning fields for human health monitoring, it became possible to enable monitoring of personal motion and vital signs in a manner that minimizes the disruption of an individual’s daily routine and assist individuals with difficulties to live independently at home. A primary difficulty that researchers confront is acquiring an adequate amount of labeled data for model training and validation purposes. Therefore, activity discovery handles the problem that activity labels are not available using approaches based on sequence mining and clustering. In this paper, we introduce an unsupervised method for discovering activities from a network of motion detectors in a smart home setting. First, we present an intra-day clustering algorithm to find frequent sequential patterns within a day. As a second step, we present an inter-day clustering algorithm to find the common frequent patterns between days. Furthermore, we refine the patterns to have more compressed and defined cluster characterizations. Finally, we track the occurrences of various regular routines to monitor the functional health in an individual’s patterns and lifestyle. We evaluate our methods on two public data sets captured in real-life settings from two apartments during seven-month and three-month periods.