Activity recognition using a supervised non-parametric hierarchical HMM

Abstract

The problem of classifying human activities occurring in depth image sequences is addressed. The 3D joint positions of a human skeleton and the local depth image pattern around these joint positions define the features. A two level hierarchical Hidden Markov Model (H-HMM), with independent Markov chains for the joint positions and depth image pattern, is used to model the features. The states corresponding to the H-HMM bottom level characterize the granular poses while the top level characterizes the coarser actions associated with the activities. Further, the H-HMM is based on a Hierarchical Dirichlet Process (HDP), and is fully non-parametric with the number of pose and action states inferred automatically from data. This is a significant advantage over classical HMM and its extensions. In order to perform classification, the relationships between the actions and the activity labels are captured using multinomial logistic regression. The proposed inference procedure ensures alignment of actions from activities with similar labels. Our construction enables information sharing, allows incorporation of unlabelled examples and provides a flexible factorized representation to include multiple data channels. Experiments with multiple real world datasets show the efficacy of our classification approach.

Simultaneous Indoor Tracking and Activity Recognition Using Pyroelectric Infrared Sensors

“…The coding scheme of the PIR sensors will be further investigated, and the reference structure will be redesigned to facilitate the data association problem of multiple targets. More sophisticated and hierarchical classification models such as conditional random field (CRF) [41] and HMM [42] will be employed to model the sequential constraints of successive activities.…”

Recognition of Activities of Daily Living via Hierarchical Long-Short Term Memory Networks

“…Many researchers modelled human motion as a stochastic process where the movement can be seen as a sequence of successive states. Inspired by the efficiency of the popular Hidden Markov Model (HMM), a hierarchical extension has been proposed in [27] to handle the motion complexity of activities. The authors proposed a two-level HMM where actions are modelled as a succession of poses and activities as successive actions states.…”

Dynamic Context-Aware Event Recognition Based on Markov Logic Networks

“…Generally, activities are mapped to the hidden states and sensory readings are mapped to the observable states in the HMM-based approaches for event recognition [17,18,19]. The HMM has many extensions, such as the coupled HMM [20], hierarchical HMM [21] and parallel HMM [22]. Similar to the HMM, the conditional random field (CRF) [23] is adopted to estimate the most likely sequence of hidden states (activities) based on the known sequence of observable states (sensing data).…”