Real-Time Activity Recognition With Instantaneous Characteristic Features of Thigh Kinematics

Abstract

Current supervised learning or deep learning-based activity recognition classifiers can achieve high accuracy in recognizing locomotion activities. Most available techniques use a high-dimensional space of features, e.g., combinations of EMG, kinematics and kinetics, and transformations over those signals.The associated classification rules are therefore complex; the machine tries to understand the human, but the human does not understand the machine. This paper presents an activity recognition system that uses signals from a thigh-mounted IMU and a force sensitive resistor to classify transitions between sitting, walking, stair ascending, and stair descending. The system uses the thigh's orientation and velocity with foot contact information at specific moments within a given activity as the features to classify transitions to other activities. We call these Instantaneous Characteristic Features (ICFs). Because these ICFs are biomechanically intuitive, they are easy for the user to understand and thus control the activity transitions of wearable robots. We assessed our classification algorithm offline using an existing dataset with 10 able-bodied subjects and online with another 10 able-bodied subjects wearing a real-time system. The offline study analyzed the effect of subject-dependency and ramp inclinations. The real-time classification accuracy was evaluated before and after training the subjects on the ICFs. The real-time system achieved overall pre-subject-training and post-subjecttraining error rates of 0.59% ± 0.24% and 0.56% ± 0.20%, respectively. We also evaluated the feasibility of our ICFs for amputee ambulation by analyzing a public dataset with the open-source bionic leg. The simplicity of these classification rules demonstrates a new paradigm for activity recognition where the human can understand the machine and vice-versa.

Data-Driven Variable Impedance Control of a Powered Knee–Ankle Prosthesis for Adaptive Speed and Incline Walking

“…This is particularly problematic during discrete transitions between steep inclines and level ground walking because the heelstrike kinematics vary drastically [24]. Future work involving anticipatory algorithms that update the task estimate based on sensed characteristics of the upcoming terrain [76]- [78] or user behavior [79], [80] may be necessary to alleviate this limitation.…”

Real-Time Human Activity Recognition with IMU and Encoder Sensors in Wearable Exoskeleton Robot via Deep Learning Networks

“…These results demonstrate that real-time HAR could be performed for the wearable robot using a standalone system. Among the mentioned HAR works based on edge devices, the best approach was recently addressed in [24]. In this study, a HAR approach was reported based on a Raspberry Pi 3 board as an application for a wearable robot or leg prosthesis.…”

“…These mentioned approaches were tested using a Raspberry Pi 3 board with embedded lightweight machine classification models such as k-nearest neighbors (KNN), convolutional neural networks (CNN), and recurrent neural networks (RNN). Despite the significant use of deep learning models for activity recognition, few studies still use traditional machine-learning methods for real-time HAR environments in edge devices [24]. More than three IMU sensors were used to recognize the lower and upper limb activities such as walk, run, jog, and open doors.…”

Data-Driven Variable Impedance Control of a Powered Knee-Ankle Prosthesis for Adaptive Speed and Incline Walking

“…This is particularly problematic during discrete transitions between steep inclines and level ground walking because the heelstrike kinematics vary drastically [24]. Future work involving anticipatory algorithms that update the task estimate based on sensed characteristics of the upcoming terrain [75]- [77] or user behavior [78], [79] may be necessary to alleviate this limitation.…”