SenseHAR

Abstract

Modern smartphones and smartwatches are equipped with inertial sensors (accelerometer, gyroscope, and magnetometer) that can be used for Human Activity Recognition (HAR) to infer tasks such as daily activities, transportation modes and, gestures. HAR requires collecting raw inertial sensor values and training a machine learning model on the collected data. The challenge in this approach is that the models are trained for specific devices and device configurations whereas, in reality, the set of devices carried by a person may vary over time. Ideally, one would like activity inferencing to be robust of this variation and provide accurate predictions by making opportunistic use of information from available devices. Moreover, the devices may be located at different parts of the body (e.g. pocket, left and right wrist), may have different sets of sensors (e.g. a smartwatch may not have gyroscope while a smartphone might), and may differ in sampling frequencies. In this paper, we provide a solution which makes use of the information from available devices while being robust to their variations. Instead of training an endto-end model for every permutation of device combinations and configurations, we propose a scalable deep learning based solution in which each device learns its own sensor fusion model that maps the raw sensor values to a shared low dimensional latent space which we call the 'SenseHAR'-a virtual activity sensor. The virtual sensor has the same format and similar behavior regardless of the subset of devices, sensor's availability, sampling rate, or a device's location. This would help machine learning engineers to develop their application specific (e.g., from gesture recognition to activities of daily life) models in a hardware-agnostic manner based on this virtual activity sensor. Our evaluations show that an application model trained on SenseHAR achieves the state of the art accuracies of 95.32%, 74.22% and 93.13% on PAMAP2, Opportunity(gestures) and our collected datasets respectively.

Semi-supervised Multi-source Domain Adaptation in Wearable Activity Recognition

“…[2] considers multiple source domains, and instead of simultaneous processing, explicitly selects the most relevant domain from the multiple source domains based on the cosine similarity with the target domain and uses the selected domain for the domain adaptation process. Whereas SenseHAR [5] proposes a data-fusion-based approach to mitigate the heterogeneous data distribution and assign labels to the unlabeled data. Authors of [5] combine multiple sensor data so that each sensor data can complement the other in achieving the intended tasks.…”

“…Whereas SenseHAR [5] proposes a data-fusion-based approach to mitigate the heterogeneous data distribution and assign labels to the unlabeled data. Authors of [5] combine multiple sensor data so that each sensor data can complement the other in achieving the intended tasks. [6] proposes an adversarial-based approach to tackle multi-source domain adaptation where multiple source domains are processed concurrently and selects the relevant source domain with the target domain using the noble perplexity scoring mechanism.…”

“…Here the first and second term determines the domain confusion loss for the source samples and target samples, respectively. Domain confusion loss for both source and target domain samples is calculated using the equation (5).…”

A Trainable Open-Source Machine Learning Accelerometer Activity Recognition Toolbox: Deep Learning Approach

“…Deep learning algorithms have gained importance in classifying human behavior based on sensor data collected from accelerometers, gyroscopes, and magnetometers [ 11 - 18 ] (for a deeper understanding and comprehensive overview, see [ 19 ]). These algorithms are based on artificial neural networks, and specifically, deep neural networks (DNNs) have become the dominant approach for activity recognition as of 2022.…”