A perspective on human activity recognition from inertial motion data

Abstract

Human activity recognition (HAR) using inertial motion data has gained a lot of momentum in recent years both in research and industrial applications. From the abstract perspective, this has been driven by the rapid dynamics for building intelligent, smart environments, and ubiquitous systems that cover all aspects of human life including healthcare, sports, manufacturing, commerce, etc., which necessitate and subsume activity recognition aiming at recognizing the actions, characteristics, and goals of one or more agent(s) from a temporal series of observations streamed from one or more sensors. From a more concrete and seemingly orthogonal perspective, such momentum has been driven by the ubiquity of inertial motion sensors on-board mobile and wearable devices including smartphones, smartwatches, etc. In this paper we give an introductory and a comprehensive survey to the subject from a given perspective. We focus on a subset of topics, that we think are major, that will have significant and influential impacts on the future research and industrial-scale deployment of HAR systems. These include: (1) a comprehensive and detailed description of the inertial motion benchmark datasets that are publicly available and/or accessible, (2) feature selection and extraction techniques and the corresponding learning methods used to build workable HAR systems; we survey classical handcrafted datasets as well as data-oriented automatic representation learning approach to the subject, (3) transfer learning as a way to overcome many hurdles in actual deployments of HAR systems on a large scale, (4) embedded implementations of HAR systems on mobile and/or wearable devices, and finally (5) we touch on adversarial attacks, a topic that is essentially related to the security and privacy of HAR systems. As the field is very huge and diverse, this article is by no means comprehensive; it is though meant to provide a logically and conceptually rather complete picture to advanced practitioners, as well as to present a readable guided introduction to newcomers. Our logical and conceptual perspectives mimic the typical data science pipeline for state-of-the-art AI-based systems.

Towards Flexible and Low-Power Wireless Smart Sensors: Reconfigurable Analog-to-Feature Conversion for Healthcare Applications

“…These constraints include a restricted variety of ADLs and low representativeness due to ranges of values, such as stride frequency and stair step height, limited by the experiment conditions. Additionally, we must address multiple heterogeneities arising from various possible software, hardware, and data collection configurations in inertial motion-based HAR systems [24]. At the same time, the signals in the UCI-HAR dataset have been collected from a particular body location by a specific smartphone with a specific operating system and embedded sensors.…”

“…Like arrhythmia detection, motion sensor-based HAR systems perform four principal steps, but one may require additional pre-processing between denoising and feature extraction, e.g., to separate body and gravitational acceleration [24]. The three primary feature extraction and selection methods are classical hand-crafted features, automatic feature generation through DL methods, and their hybrid.…”

Recent Advances in Wearable Healthcare Devices: From Material to Application

“…Additionally, wearable IMUs are used to monitor the safety behaviors of scaffolding workers [188], enhance human location with Wi-Fi sensing [189,190], count exercise repetition [191], and assessment of knee moments [192]. More applications of IMUs can be found in these reviews [193][194][195][196][197][198][199].…”

A Novel Framework Based on Deep Learning Architecture for Continuous Human Activity Recognition with Inertial Sensors

“…There exist two main categories in which to classify the type of sensors: fixed sensors (e.g., videocameras, proximity and light sensors) are installed at specific locations of the environment and monitor activities in a confined area, whereas wearable sensors are directly worn by the subject, as in the case of inertial measurement units (IMUs), pressure and heart rate sensors [5]. Though being previously utilized to accurately label activities [6,15], fixed sensors like cameras are not very suitable when ADLs execution requires subjects to move outside the area covered by them [24]; besides, cameras suffer from variable illumination, occlusion occurrence, presence of shadows, and time-varying contrast, especially in outdoor environments; such disadvantages, together with privacy issues and their lack of portability prevent them from continuously monitoring human activities [33]. In light of these limitations, most of the research in the HAR field, especially for remote monitoring, has preferably adopted wearable sensors because of their low cost and higher flexibility in providing continuous monitoring [34].…”