Policy Recognition in the Abstract Hidden Markov Model

Abstract

In this paper, we present a method for recognising an agent's behaviour in dynamic, noisy, uncertain domains, and across multiple levels of abstraction. We term this problem on-line plan recognition under uncertainty and view it generally as probabilistic inference on the stochastic process representing the execution of the agent's plan. Our contributions in this paper are twofold. In terms of probabilistic inference, we introduce the Abstract Hidden Markov Model (AHMM), a novel type of stochastic processes, provide its dynamic Bayesian network (DBN) structure and analyse the properties of this network. We then describe an application of the Rao-Blackwellised Particle Filter to the AHMM which allows us to construct an efficient, hybrid inference method for this model. In terms of plan recognition, we propose a novel plan recognition framework based on the AHMM as the plan execution model. The Rao-Blackwellised hybrid inference for AHMM can take advantage of the independence properties inherent in a model of plan execution, leading to an algorithm for online probabilistic plan recognition that scales well with the number of levels in the plan hierarchy. This illustrates that while stochastic models for plan execution can be complex, they exhibit special structures which, if exploited, can lead to efficient plan recognition algorithms. We demonstrate the usefulness of the AHMM framework via a behaviour recognition system in a complex spatial environment using distributed video surveillance data.

Action understanding as inverse planning

“…These approaches use sophisticated, hierarchical representations of goals and subtasks, such as scripts and event hierarchies, to model the structure of agents' behavior, and model goal inference in terms of logical sufficiency or necessity of the observed behavior for achieving a particular goal. Probabilistic versions of these ideas have also been proposed, which allow inductive, graded inferences of structured goals and plans from observations of behavior (Charniak & Goldman, 1991;Bui et al, 2002;Liao et al, 2004). However, these approaches assume that the distribution over actions, conditioned on goals, is either available a priori (Charniak & Goldman, 1991;Bui et al, 2002), or must be estimated from a large dataset of observed actions (Liao et al, 2004).…”

“…Probabilistic versions of these ideas have also been proposed, which allow inductive, graded inferences of structured goals and plans from observations of behavior (Charniak & Goldman, 1991;Bui et al, 2002;Liao et al, 2004). However, these approaches assume that the distribution over actions, conditioned on goals, is either available a priori (Charniak & Goldman, 1991;Bui et al, 2002), or must be estimated from a large dataset of observed actions (Liao et al, 2004). An alternative is to model the abstract principles underlying intentional action, which can be used to generate action predictions in novel situations, without requiring a large dataset of prior observations.…”

Opportunity Knocks: A System to Provide Cognitive Assistance with Transportation Services

“…The switching nodes f g k , f t k and f m k indicate when changes in a variable's value can happen. An efficient algorithm based on Rao-Blackwellised particle filters [14][15][16] has been developed to perform online inference for this model. At the lowest level, location tracking on the street map is done using graph-based Kalman filtering that is more efficient than the grid-based Bayesian filter and traditional particle filtering [17], used for the 2TBN model.…”

“…Bui, et al [15] introduced the abstract hidden Markov model which uses hierarchical representations to efficiently infer a person's goal in an indoor environment from camera information. Later, Bui [16] extended this model to include memory nodes, which enables the transfer of context information over multiple time steps.…”

Detecting malicious groups of agents

“…Most previous work on plan and goal recognition has assumed cooperative or neutral agents [22,33,7,6,21]. Cohen, Perrault and Allen [8] distinguish between two kinds of plan recognition: keyhole and intended.…”