Triple Cross-Domain Attention on Human Activity Recognition Using Wearable Sensors

Tang, Yin; Zhang, Lei; Teng, Qi; Min, Fuhong; Song, Aiguo

doi:10.1109/tetci.2021.3136642

Cited by 65 publications

(25 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our benchmark regroups head-to-head 17 articles (21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35) by sharing their approach concerning the human activity recognition task evaluated on the UniMiB-SHAR dataset. Deep learning was the method of choice in almost every case (21-26, 28-32, 34, 35) to try to achieve state-of-the-art results.…”

Section: Related Work From Benchmarkmentioning

confidence: 99%

“…Second, some authors mentioned that their deep learning architecture should be kept reasonable in terms of memory consumption and computation time (23,24,26,29,30). This is only logical from an operational perspective, as the final goal of the algorithm is to be embedded into a portable device and to run live, which some studies have tested with their solutions (21,26,28,29). The question of feature representation in the case of human activity recognition was also raised as is often the case in deep learning studies.…”

Section: Related Work From Benchmarkmentioning

confidence: 99%

“…This means that the results and metrics reported by one study cannot always be rigorously compared with all the others, as they must use the same evaluation methods. The evaluation methods featured in these articles are as follows: the evaluation of a validation set (21)(22)(23)(24)(25)(26), the evaluation of a test set (27, 28), cross-validation (29)(30)(31)(32), and leave-one-subject-out (32,34,35).…”

Section: Related Work From Benchmarkmentioning

confidence: 99%

See 2 more Smart Citations

The use of deep learning for smartphone-based human activity recognition

Stampfler

Elgendi²,

Fletcher³

et al. 2023

Front. Public Health

View full text Add to dashboard Cite

The emerging field of digital phenotyping leverages the numerous sensors embedded in a smartphone to better understand its user's current psychological state and behavior, enabling improved health support systems for patients. As part of this work, a common task is to use the smartphone accelerometer to automatically recognize or classify the behavior of the user, known as human activity recognition (HAR). In this article, we present a deep learning method using the Resnet architecture to implement HAR using the popular UniMiB-SHAR public dataset, containing 11,771 measurement segments from 30 users ranging in age between 18 and 60 years. We present a unified deep learning approach based on a Resnet architecture that consistently exceeds the state-of-the-art accuracy and F1-score across all classification tasks and evaluation methods mentioned in the literature. The most notable increase we disclose regards the leave-one-subject-out evaluation, known as the most rigorous evaluation method, where we push the state-of-the-art accuracy from 78.24 to 80.09% and the F1-score from 78.40 to 79.36%. For such results, we resorted to deep learning techniques, such as hyper-parameter tuning, label smoothing, and dropout, which helped regularize the Resnet training and reduced overfitting. We discuss how our approach could easily be adapted to perform HAR in real-time and discuss future research directions.

show abstract

Section: Related Work From Benchmarkmentioning

confidence: 99%

Section: Related Work From Benchmarkmentioning

confidence: 99%

Section: Related Work From Benchmarkmentioning

confidence: 99%

See 1 more Smart Citation

The use of deep learning for smartphone-based human activity recognition

Stampfler

Elgendi²,

Fletcher³

et al. 2023

Front. Public Health

View full text Add to dashboard Cite

show abstract

“…Besides, a shallow CNN to consider cross-channel interaction within the human activities is proposed [30]. A triplet cross-dimension attention for HAR is proposed to capture the interaction between three dimensions, i.e., sensor, temporal and channel [31]. These methods have reported state-of-theart performance, however, they have only validated by the wearable sensors data which are less imbalanced.…”

Section: Related Workmentioning

confidence: 99%

Cross-Domain Activity Recognition Using Shared Representation in Sensor Data

et al. 2022

View full text Add to dashboard Cite

Existing models based on sensor data for human activity recognition are reporting state-of-the-art performances. Most of these models are conducted based on single-domain learning in which for each domain a model is required to be trained. However, the generation of adequate labelled data and a learning model for each domain separately is often time-consuming and computationally expensive. Moreover, the deployment of multiple domainwise models is not scalable as it obscures domain distinctions, introduces extra computational costs, and limits the usefulness of training data. To mitigate this, we propose a multi-domain learning network to transfer knowledge across different but related domains and alleviate isolated learning paradigms using a shared representation. The proposed network consists of two identical causal convolutional sub-networks that are projected to a shared representation followed by a linear attention mechanism. The proposed network can be trained using the full training dataset of the source domain and a dataset of restricted size of the target training domain to reduce the need of large labelled training datasets. The network processes the source and target domains jointly to learn powerful and mutually complementary features to boost the performance in both domains. The proposed multi-domain learning network on six real-world sensor activity datasets outperforms the existing methods by applying only 50% of the labelled data. This confirms the efficacy of the proposed approach as a generic model to learn human activities from different but related domains in a joint effort, to reduce the number of required models and thus improve system efficiency.

show abstract

“…However, features extracted from deep neural networks are often treated as a side effect of the classifier training, rather than being explicitly sought. Metric learning methods, such as Siamese Neural Networks (SNN) [ 37 ] and Triplet Neural Networks (TNN) [ 11 , 12 , 38 ] optimize an embedding directly for the desired task. Triplet selection strategies have been proposed for domain-specific tasks, which improve performance from the naive implementation.…”

Section: Introductionmentioning

confidence: 99%

Personalized Activity Recognition with Deep Triplet Embeddings

Burns

Whyne

2022

Sensors

View full text Add to dashboard Cite

A significant challenge for a supervised learning approach to inertial human activity recognition is the heterogeneity of data generated by individual users, resulting in very poor performance for some subjects. We present an approach to personalized activity recognition based on deep feature representation derived from a convolutional neural network (CNN). We experiment with both categorical cross-entropy loss and triplet loss for training, and describe a novel loss function based on subject triplets. We evaluate these methods on three publicly available inertial human activity recognition datasets (MHEALTH, WISDM, and SPAR) comparing classification accuracy, out-of-distribution activity detection, and generalization to new activity classes. The proposed triplet algorithm achieved an average 96.7% classification accuracy across tested datasets versus the 87.5% achieved by the baseline CNN algorithm. We demonstrate that personalized algorithms, and, in particular, the proposed novel triplet loss algorithms, are more robust to inter-subject variability and thus exhibit better performance on classification and out-of-distribution detection tasks.

show abstract

Triple Cross-Domain Attention on Human Activity Recognition Using Wearable Sensors

Cited by 65 publications

References 30 publications

The use of deep learning for smartphone-based human activity recognition

The use of deep learning for smartphone-based human activity recognition

Cross-Domain Activity Recognition Using Shared Representation in Sensor Data

Personalized Activity Recognition with Deep Triplet Embeddings

Contact Info

Product

Resources

About