Activity2Vec: Learning ADL Embeddings from Sensor Data with a Sequence-to-Sequence Model

Abstract

Recognizing activities of daily living (ADLs) plays an essential role in analyzing human health and behavior. The widespread availability of sensors implanted in homes, smartphones, and smart watches have engendered collection of big datasets that reflect human behavior. To obtain a machine learning model based on these data,researchers have developed multiple feature extraction methods. In this study, we investigate a method for automatically extracting universal and meaningful features that are applicable across similar time series-based learning tasks such as activity recognition and fall detection. We propose creating a sequence-tosequence (seq2seq) model to perform this feature learning. Beside avoiding feature engineering, the meaningful features learned by the seq2seq model can also be utilized for semi-supervised learning. We evaluate both of these benefits on datasets collected from wearable and ambient sensors. CCS CONCEPTS• Theory of computation → Machine learning theory; Models of learning; Unsupervised learning and clustering; • Computing methodologies → Neural networks.

A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning

“…Ghods et al [44] proposed a method, Activity2Vec to learn an activity Embedding from sensor data. They used a Sequence-to-Sequence model (Seq2Seq) [45] to encode and extract automatic features from sensors.…”

ActSafe: Predicting Violations of Medical Temporal Constraints for Medication Adherence

“…MTC types 3 and 5 could not be evaluated in this experiment due to lack of instances in the prescription dataset. [9,20,22] and thus rely on differing inputs, comparing the performance of HERBERT with these models is not fair. Hence, we consider only statistical models as our baselines for time series prediction.…”

“…Predictive modeling of human behavior is a burgeoning field, with an increased focus on sensor-based activity recognition models [20,22,32]. There are three main formulations of the sensor-based activity prediction task.…”

The Lifespan of Human Activity Recognition Systems for Smart Homes

“…The methods developed in [11] have been tested on some of the collected CASAS datasets but not on others; thus, it is not clear if the given method generalizes across different smart home settings. The works in [8,9] developed methods for specific homes [5] by making use of object and appliance interactions (developed through ontologies) that do not generalize to other smart homes, which consist of different objects.…”

“…were used on event-based data instances CASAS datasets (Milan, Cairo, Kyoto7, Kyoto8, and Kyoto11) [30] Fully convolutional network bootstrapped by word encoding and embedding for activity recognition in smart homes [89] Event-based analysis; requires annotated data A word2vec encoding was applied to sensor event-based windows, which were then passed through a fully convolutional network for classification CASAS datasets (Aruba and Milan) [30] Using the language model to bootstrap human activity recognition that utilized ambient sensors Based in smart homes [90] Event-based analysis; requires annotated data Different embedding techniques were used to obtain the learned features followed by a sequential modeling procedure (LSTM) on the event-based data instances CASAS datasets (Aruba, Milan, and Cairo) [30] Activity2vec: Learning adl embeddings from sensor data with a sequence-to-sequence model [11] Requires annotated data A sequence-to-sequence model was used to generate features followed by a random forest model for classification CASAS dataset (HH101) [30] Enhancing activity recognition using CPD-based activity segmentation [71] Requires annotated data A heuristic function followed by a dissimilarity-based approach were used to identify change points. Handcrafted features were extracted.…”