Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data

Abstract

Advances in deep learning for human activity recognition have been relatively limited due to the lack of large labelled datasets. In this study, we leverage selfsupervised learning techniques on the UK-Biobank activity tracker dataset-the largest of its kind to date-containing more than 700,000 person-days of unlabelled wearable sensor data. Our resulting activity recognition model consistently outperformed strong baselines across seven benchmark datasets, with an F1 relative improvement of 2.5%-100% (median 18.4%), the largest improvements occurring in the smaller datasets. In contrast to previous studies, our results generalise across external datasets, devices, and environments. Our open-source model will help researchers and developers to build customisable and generalisable activity classifiers with high performance.

Digital health technologies and machine learning augment patient reported outcomes to remotely characterise rheumatoid arthritis

“…Two RA participants withdrew immediately after enrolling in the study. Data from these participants were not collected, leaving 28 RA participants, 28 matched HCs, and 2 unmatched HCs for a total of 58 participant Assessing smartwatch-based daily physical activity patterns The daily physical activity of RA participants and healthy controls were estimated with a deep convolutional neural network (DCNN) that was first pre-trained on 100,000 participants in the publicly available UK Biobank, following a multi-task self-supervised learning (SSL) methodology 24 , which was subsequently fine-tuned on the free-living Capture-24 dataset of <150 participants 25 . In this study, we build upon our previous work by adding a temporal dependency to the DCNN (SSL) through a hidden markov model (HMM), which was appended to obtain a more accurate sequence of predicted activities over the continuous study period.…”

“…A deep convolutional neural network (DCNN) with a ResNet-V2 architecture was first pre-trained following a multitask self-supervised learning (SSL) methodology on 100,000 participants—each participant contributing 7 days yielding roughly 700,000 person days of data—in the open-source UK biobank 24 . The SSL pre-trained model was then fine-tuned to perform activity recognition as a downstream task in the Capture-24 dataset.…”

“…Our multi-task SSL training relied on the unlabelled UKB , which contains roughly 700,000 person-days of free-living activity data (100,000 participants, 7 days of wear). For more information we refer the reader to our previous work 24 .…”

“…Deep networks, in particular, need a lot of training data in order to be robust and generaliseable. Open-source HAR-based datasets have small sample sizes, with generally n<100 participants as annotating free-living wearable data for human activity recognition (HAR) requires a concurrent video stream, and the labelling process is resource-intensive 24 .…”

“…The Capture-24 fine-tuned HAR SSL-HMM model was then implemented to estimate daily activities in weaRAble-PRO study data. For additional information of the HAR deep network, SSL, and other related information, we refer the reader to our previous work 24 . Further results relating to the SSL-HMM model are outlined in the supplementary material.…”