Assessing the State of Self-Supervised Human Activity Recognition using Wearables

Abstract

The emergence of self-supervised learning in the field of wearables-based human activity recognition (HAR) has opened up opportunities to tackle the most pressing challenges in the field, namely to exploit unlabeled data to derive reliable recognition systems from only small amounts of labeled training samples. Furthermore, self-supervised methods enable a host of new application domains such as, for example, domain adaptation and transfer across sensor positions, activities etc. As such, self-supervision, i.e., the paradigm of 'pretrain-then-finetune' has the potential to become a strong alternative to the predominant end-to-end training approaches, let alone the classic activity recognition chain with explicit focus on designing feature representations for sensor data. Recently a number of contributions have been made that introduced self-supervised learning into the field of HAR, including, Multi-task self-supervision, Masked Reconstruction, CPC to name but a few. With the initial success of these methods, the time has come for a systematic inventory and analysis of the potential self-supervised learning has for the field. This paper provides exactly that. We assess the progress of self-supervised HAR research by introducing a framework that performs a multi-faceted exploration of model performance. We organize the framework into three dimensions, each containing three constituent criteria, and utilize it to assess state-of-the-art self-supervised learning methods in a large empirical study on a curated set of nine diverse benchmarks. This exploration leads us to the formulation of insights into the properties of these techniques and to establish their value towards learning representations for diverse scenarios. Based on our findings we call upon the community to join our efforts and to contribute towards shaping the evaluation of the ongoing paradigm change in modeling human activities from body-worn sensor data.CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing; Empirical studies in ubiquitous and mobile computing; • Computing methodologies → Machine learning.

Combining Public Human Activity Recognition Datasets to Mitigate Labeled Data Scarcity

“…In recent years, self-supervised learning (SSL) approaches have gained attention as a general framework for learning from unlabeled data through a pretext task [5], [32]. SSL has also been applied to sensor-based HAR, leading to promising results [33], [34]. However, even a good unsupervised or SSL model that permits adaptation using a short amount of data still requires a significant amount of data, as exemplified in other domains such as vision and speech [35].…”

Self-supervised learning for early detection of neurodegenerative diseases with small data

“…Modeling the heterogeneous PD symptoms is challenging and we could be better served modeling the physiological patterns of healthy subjects instead. The validity of our approach has been corroborated in recent work [361] which benchmarked the various self-supervised learning algorithms when used on the task of activity recognition. Haresamudram et al [361] found that increasing the number of participants intuitively improves downstream performance.…”

“…The validity of our approach has been corroborated in recent work [361] which benchmarked the various self-supervised learning algorithms when used on the task of activity recognition. Haresamudram et al [361] found that increasing the number of participants intuitively improves downstream performance. However, the gains quickly saturate with no significant difference between using 25% and 100% of participants.…”