Advanced visual surveillance using Bayesian networks

Abstract. We present a comparative analysis of a layered architecture of Hidden Markov Models (HMMs) and dynamic Bayesian networks (DBNs) for identifying human activites from multimodal sensor information. We use the two representations to diagnose users' activities in S-SEER, a multimodal system for recognizing office activity from realtime streams of evidence from video, audio and computer (keyboard and mouse) interactions. As the computation required for sensing and processing perceptual information can impose significant burdens on personal computers, the system is designed to perform selective perception using expected-value-of-information (EVI) to limit sensing and analysis. We discuss the relative performance of HMMs and DBNs in the context of diagnosis and EVI computation.

show abstract

“…DBNs have been adopted by several researchers for the modeling and recognition of human activities [3,5,10].…”

Section: Prior Related Work On Human Activity Recognitionmentioning

confidence: 99%

A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities

Oliver

Horvitz

2005

User Modeling 2005

View full text Add to dashboard Cite

show abstract

“…Dynamic Bayesian networks have been adopted by several researchers for the modeling and recognition of human activities [10][11][12][13][14][15].…”

Section: Prior Related Workmentioning

confidence: 99%

S-SEER: Selective Perception in a Multimodal Office Activity Recognition System

Oliver

Horvitz

2005

Machine Learning for Multimodal Interaction

View full text Add to dashboard Cite

Abstract. The computation required for sensing and processing perceptual information can impose significant burdens on personal computer systems. We explore several policies for selective perception in SEER, a multimodal system for recognizing office activity that relies on a cascade of Hidden Markov Models (HMMs) named Layered Hidden Markov Model (LHMMs). We use LHMMs to diagnose states of a user's activity based on real-time streams of evidence from video, audio and computer (keyboard and mouse) interactions. We review our efforts to employ expected-value-of-information (EVI) to limit sensing and analysis in a context-sensitive manner. We discuss an implementation of a greedy EVI analysis and compare the results of using this analysis with a heuristic sensing policy that makes observations at different frequencies. Both policies are then compared to a random perception policy, where sensors are selected at random. Finally, we discuss the sensitivity of ideal perceptual actions to preferences encoded in utility models about information value and the cost of sensing.

show abstract

“…One of the earlier examples of using Dynamic Belief networks (DBN) for visual surveillance appears in [5]. DBNs offer many advantages for tracking tasks such as incorporation of prior knowledge and good modelling ability to represent the dynamic dependencies between parameters involved in a visual interpretation.…”

Section: Visual Recognition As Perceptual Inferencementioning

confidence: 99%

A Self-Referential Perceptual Inference Framework for Video Interpretation

Town

Sinclair²

2003

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper presents an extensible architectural model for general content-based analysis and indexing of video data which can be customised for a given problem domain. Video interpretation is approached as a joint inference problems which can be solved through the use of modern machine learning and probabilistic inference techniques. An important aspect of the work concerns the use of a novel active knowledge representation methodology based on an ontological query language. This representation allows one to pose the problem of video analysis in terms of queries expressed in a visual language incorporating prior hierarchical knowledge of the syntactic and semantic structure of entities, relationships, and events of interest occurring in a video sequence. Perceptual inference then takes place within an ontological domain defined by the structure of the problem and the current goal set.

show abstract

Advanced visual surveillance using Bayesian networks

Cited by 42 publications

References 23 publications

A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities

A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities

S-SEER: Selective Perception in a Multimodal Office Activity Recognition System

A Self-Referential Perceptual Inference Framework for Video Interpretation

Contact Info

Product

Resources

About