Efficient duration and hierarchical modeling for human activity recognition

Abstract

A challenge in building pervasive and smart spaces is to learn and recognize human activities of daily living (ADLs). In this paper, we address this problem and argue that in dealing with ADLs, it is beneficial to exploit both their typical duration patterns and inherent hierarchical structures. We exploit efficient duration modeling using the novel Coxian distribution to form the Coxian hidden semi-Markov model (CxHSMM) and apply it to the problem of learning and recognizing ADLs with complex temporal dependencies. The Coxian duration model has several advantages over existing duration parameterization using multinomial or exponential family distributions, including its denseness in the space of nonnegative distributions, low number of parameters, computational efficiency and the existence of closed-form estimation solutions. Further we combine both hierarchical and duration extensions of the hidden Markov model (HMM) to form the novel switching hidden semi-Markov model (SHSMM), and empirically compare its performance with existing models. The model can learn what an occupant normally does during the day from unsegmented training data and then perform online activity classification, segmentation and abnormality detection. Experimental results show that Coxian modeling outperforms a range of baseline models for the task of activity segmentation. We also achieve a recognition accuracy competitive to the current state-of-the-art multinomial duration model, while gaining a significant reduction in computation. Furthermore, cross-validation model selection on the number of phases K in the Coxian indicates that only a small K is required to achieve the optimal performance. Finally, our models are further tested in a more challenging setting in which the tracking is often lost and the activities considerably overlap. With a small amount of labels supplied during training in a partially supervised learning mode, our models are again able to deliver reliable performance, again with a small number of phases, making our proposed framework an attractive choice for activity modeling.

The Design of a Proactive Personal Agent for Task Management

“…This allows CALO both to identify the most likely step and to identify the most likely parameter values for this step. The algorithm is described further by Duong et al 21 ; it has been extended to handle multiple interleaved concurrent workflows. The state information from the Workflow Tracker is provided to the Execution Monitor ( Figure 1) and exploited by the assistance patterns to generate situationally relevant Candidate Goals, both task focused (i.e., relating to the task that Workflow Tracker identifies the user is working on) and utility focused (e.g., relating to overall workload).…”

Complete Sound and Speech Recognition System for Health Smart Homes: Application to the Recognition of Activities of Daily Living

“…Within the PROSAFE project, the ERGDOM system controls the comfort of the person inside the flat (i.e., temperature, light...). Regarding the activity detection, although most of the many researches related to health smart homes is focused on sensors, network and data sharing (Chan et al, 2008), a fair number of laboratories started to work on reliable Activities of Daily Living (ADL) detection and classification using Bayesian (Dalal et al, 2005), rule-based (Duong et al, 2009;Moore & Essa, 2002), evidential fusion (Hong et al, 2008), Markovian (Albinali et al, 2007;Kröse et al, 2008), Support Vector Machine (Fleury, 2008), or ensemble of classifiers (Albinali et al, 2007) approaches. For instance, (Kröse et al, 2008) learned models to recognize two activities: 'going to the toilets' and 'exit from the flat'.…”

“…One of the first steps to achieve these goals is to detect the daily activities and to assess the evolution of the monitored person's autonomy. Therefore, activity recognition is an active research area (Albinali et al, 2007;Dalal et al, 2005;Duchêne et al, 2007;Duong et al, 2009;Fleury, 2008;Moore & Essa, 2002) but, despite this, it has still not reached a satisfactory performance nor led to a standard methodology. One reason is the high number of flat configurations and available sensors (e.g., infra-red sensors, contact doors, video cameras, RFID tags, etc.)…”

Everyday Life Sounds Database: Telemonitoring of Elderly or Disabled

“…Activity recognition is an active research area [1,[4][5][6]8]; however, despite this, it has not yet reached a satisfactory performance or resulted in a standard method [26]. Among the objectives of this area is the recognition of distress situations among the elderly or disabled persons in their habitats, for the purpose of their surveillance.…”