Contrastive Predictive Coding for Human Activity Recognition

Abstract

Feature extraction is crucial for human activity recognition (HAR) using body-worn movement sensors. Recently, learned representations have been used successfully, offering promising alternatives to manually engineered features. Our work focuses on effective use of small amounts of labeled data and the opportunistic exploitation of unlabeled data that are straightforward to collect in mobile and ubiquitous computing scenarios. We hypothesize and demonstrate that explicitly considering the temporality of sensor data at representation level plays an important role for effective HAR in challenging scenarios. We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams. Through a range of experimental evaluations on real-life recognition tasks, we demonstrate its effectiveness for improved HAR. CPC-based pre-training is self-supervised, and the resulting learned representations can be integrated into standard activity chains. It leads to significantly improved recognition performance when only small amounts of labeled training data are available, thereby demonstrating the practical value of our approach.

Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data

“…Recent studies explored the use of SSL for HAR, such as forecasting (Taghanaki and Etemad, 2020), masked reconstruction (Haresamudram et al, 2020a), contrastive learning (Haresamudram et al, 2020b;Tang et al, 2020), and multi-task SSL (Saeed et al, 2019;Tang et al, 2021). However, these studies were still limited by small laboratory-style datasets (n < 10,000), which confounded their findings.…”