Confidence-Calibrated Human Activity Recognition

Abstract

Wearable sensors are widely used in activity recognition (AR) tasks with broad applicability in health and well-being, sports, geriatric care, etc. Deep learning (DL) has been at the forefront of progress in activity classification with wearable sensors. However, most state-of-the-art DL models used for AR are trained to discriminate different activity classes at high accuracy, not considering the confidence calibration of predictive output of those models. This results in probabilistic estimates that might not capture the true likelihood and is thus unreliable. In practice, it tends to produce overconfident estimates. In this paper, the problem is addressed by proposing deep time ensembles, a novel ensembling method capable of producing calibrated confidence estimates from neural network architectures. In particular, the method trains an ensemble of network models with temporal sequences extracted by varying the window size over the input time series and averaging the predictive output. The method is evaluated on four different benchmark HAR datasets and three different neural network architectures. Across all the datasets and architectures, our method shows an improvement in calibration by reducing the expected calibration error (ECE)by at least 40%, thereby providing superior likelihood estimates. In addition to providing reliable predictions our method also outperforms the state-of-the-art classification results in the WISDM, UCI HAR, and PAMAP2 datasets and performs as good as the state-of-the-art in the Skoda dataset.

Mixing temporal experts for Human Activity Recognition

“…In this paper, a framework is proposed that trains a gated neural network responsible for assigning weights to the outputs of the temporal experts, i.e., the pre-trained models of the ensemble. This framework is an improvement over our previous work [1], and it has also shown success in another ensemble-based method in HAR tasks [2]. Figure 1 demonstrates the framework using a sequence from running activity as an example.…”

“…Since it is a time-series classification problem, neural networks are popular in modelling the task. Our most recent work [1] and another one by Guan et al [2] achieve state-of-the-art results in this domain through the ensembling of neural network models. In both works, the temporality of the data is exploited during the training of individual models of the ensemble.…”

“…to model raw time-series data without any significant preprocessing steps. For example, in the Deep time ensembles (DTE) approach [1], each model in the ensemble become expert in recognizing temporal patterns of different length. These temporal sequences are generated by parsing the raw time-series data (in a sliding window approach) with different window lengths.…”

“…There are n temporal experts, and in this example, it is assumed that they are experts in recognizing certain types of activities such as walking, running, etc. Please note that the temporal experts are not explicitly trained to recognize only a single activity rather input sequences of different lengths [1], but for the sake of explanation, it is assumed in this example. In Figure 1, it is seen that the same input sequence (part A) (running) is fed to n pre-trained temporal experts and, to an untrained neural network called the neural gate (part B).…”

“…The framework is tested on a very popular HAR dataset called PAMAP2 [3] with the two ensembles-based modelling approach DTE [1] and LSTM ensembles [2]. The f1 score of the classification task for LSTM-ensemble [2] and for DTE [1] was 0.85 and 0.89 respectively. The framework improved the f1 score to 0.89 in the first case and 0.91 in the secondcase.…”