Out-of-Distribution Detection of Human Activity Recognition with Smartwatch Inertial Sensors

Abstract

Out-of-distribution (OOD) in the context of Human Activity Recognition (HAR) refers to data from activity classes that are not represented in the training data of a Machine Learning (ML) algorithm. OOD data are a challenge to classify accurately for most ML algorithms, especially deep learning models that are prone to overconfident predictions based on in-distribution (IIN) classes. To simulate the OOD problem in physiotherapy, our team collected a new dataset (SPARS9x) consisting of inertial data captured by smartwatches worn by 20 healthy subjects as they performed supervised physiotherapy exercises (IIN), followed by a minimum 3 h of data captured for each subject as they engaged in unrelated and unstructured activities (OOD). In this paper, we experiment with three traditional algorithms for OOD-detection using engineered statistical features, deep learning-generated features, and several popular deep learning approaches on SPARS9x and two other publicly-available human activity datasets (MHEALTH and SPARS). We demonstrate that, while deep learning algorithms perform better than simple traditional algorithms such as KNN with engineered features for in-distribution classification, traditional algorithms outperform deep learning approaches for OOD detection for these HAR time series datasets.

Special Issue “Wearable and BAN Sensors for Physical Rehabilitation and eHealth Architectures”

“…The authors collected a new data set (SPARS9x) from inertial smartwatch sensors worn by 20 volunteers, first performing supervised physical exercises and, after, performing other unrelated physical movements (OOD). From this analysis, the authors showed that traditional algorithms outperform deep learning algorithms for this particular case of OOD detection for HAR [7].…”

“…Another work devoted to the human activity recognition (HAR), using smartwatch inertial sensors, was presented by the authors of [7]. The authors study the performance of three algorithms for the out-of-distribution (OOD) detection of activity classes data that are not present in the training data of the ML [7].…”

“…Another work devoted to the human activity recognition (HAR), using smartwatch inertial sensors, was presented by the authors of [7]. The authors study the performance of three algorithms for the out-of-distribution (OOD) detection of activity classes data that are not present in the training data of the ML [7]. The authors collected a new data set (SPARS9x) from inertial smartwatch sensors worn by 20 volunteers, first performing supervised physical exercises and, after, performing other unrelated physical movements (OOD).…”

AI Technologies for Machine Supervision and Help in a Rehabilitation Scenario

“…There have been multiple works specifically aimed at assessing therapy exercises and providing feedback [25][26][27]. However, the current approaches can only provide automatic motion detection or assessment based on wearable devices or video input.…”

Personalized Activity Recognition with Deep Triplet Embeddings

“…The advantage of this methodology is that the embedding method can be fitted or trained to a large dataset in advance, while user-specific interrogation can be rapidly accomplished in a de novo fashion via feature extraction through the pre-trained embedder, with the resulting embedding subsequently used to perform classification and/or characterization. Further benefits of this approach include the capacity to incorporate novel activity classes without model re-training, and identify out-of-distribution (OOD) activity classes (i.e., samples drawn from class distributions previously unseen in classifier training), thereby supporting an open-set activity recognition framework [ 32 , 33 , 34 ].…”