Automatic Video Interpretation: A Recognition Algorithm for Temporal Scenarios Based on Pre-compiled Scenario Models

Abstract. This paper presents a cognitive vision approach to recognize a set of interesting activities of daily living (ADLs) for elderly at home. The proposed approach is composed of a video analysis component and an activity recognition component. A video analysis component contains person detection, person tracking and human posture recognition. A human posture recognition is composed of a set of postures models and a dedicated human posture recognition algorithm. Activity recognition component contains a set of video event models and a dedicated video event recognition algorithm. In this study, we collaborate with medical experts (gerontologists from Nice hospital) to define and model a set of scenarios related to the interesting activities of elderly. Some of these activities require to detect a fine description of human body such as postures. For this purpose, we propose ten 3D key human postures usefull to recognize a set of interesting human activities regardless of the environment. Using these 3D key human postures, we have modeled thirty four video events, simple ones such as "a person is standing" and composite ones such as "a person is feeling faint". We have also adapted a video event recognition algorithm to detect in real time some activities of interest by adding posture. The novelty of our approach is the proposed 3D key postures and the set of activity models of elderly person living alone in her/his own home.To validate our proposed models, we have performed a set of experiments in the Gerhome laboratory which is a realistic site reproducing the environment of a typical apartment. For these experiments, we have acquired and processed ten video sequences with one actor. The duration of each video sequence is about ten minutes and each video contains about 4800 frames.

show abstract

“…In this work we have used the approach described in [15]. This approach is based on constraint resolution.…”

Section: Activity Recognitionmentioning

confidence: 99%

“…The video event recognition algorithm is based on the method described in [15]. We have adapted this algorithm to detect in real time some activities of interest by adding posture.…”

Section: Activity Recognitionmentioning

confidence: 99%

Monitoring Activities of Daily Living (ADLs) of Elderly Based on 3D Key Human Postures

Zouba

Boulay

Brémond

et al. 2008

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…The models of scenarios of interest are the scenarios specified by the end-users. The models of scenarios are defined using a description language designed in a generic framework [15] which has been conceived for other types of video understanding applications. These scenario models are then used by the SAMSIT platform for interpreting audiovideo streams.…”

Section: The Knowledge Basementioning

confidence: 99%

Audio-video event recognition system for public transport security

Vu¹,

Brémond²,

Davini³

et al. 2006

IET Conference on Crime and Security

View full text Add to dashboard Cite

This paper presents an audio-video surveillance system for the automatic surveillance in public transport vehicle. The system comprises six modules including in particular three novel ones: (i) Face Detection and Tracking, (ii) Audio Event Detection and (iii) Audio-Video Scenario Recognition. The Face Detection and Tracking module is responsible for detecting and tracking faces of people in front of cameras. The Audio Event Detection module detects abnormal audio events which are precursor for detecting scenarios which have been predefined by end-users. The Audio-Video Scenario Recognition module performs high level interpretation of the observed objects by combining audio and video events based on spatio-temporal reasoning. The performance of the system is evaluated for a series of pre-defined audio, video and audio-video events specified using an audio-video event ontology.

show abstract

“…Then the recognition of composite states and events is performed in a similar way to the recognition of primitive events. The video event recognition algorithm is based on the method of Vu et al [12].…”

Section: Video Event Recognitionmentioning

confidence: 99%

“…Recent work by Xiang et al [14] applied a hierarchical dynamic Bayesian network to recognise scene events; however, such models are incapable of recognising simultaneous complex scene activities in real-time over extended time periods. The approach adopted for AVITRACK [12] addresses these problems using cognitive vision techniques based on spatio-temporal reasoning, a priori knowledge of the observed scene and a set of predefined video events corresponding to the servicing operations to recognise. Previous work was performed on primitive video events; here the focus is on more complex video events corresponding to servicing operations on apron area.…”

Section: Introductionmentioning

confidence: 99%

Automated Scene Understanding for Airport Aprons

Ferryman

Borg

Thirde

et al. 2005

AI 2005: Advances in Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Abstract. This paper presents a complete visual surveillance system for automatic scene interpretation of airport aprons. The system comprises two main modules -Scene Tracking and Scene Understanding. The Scene Tracking module is responsible for detecting, tracking and classifying the semantic objects within the scene using computer vision. The Scene Understanding module performs high level interpretation of the observed objects by detecting video events using cognitive vision techniques based on spatio-temporal reasoning. The performance of the system is evaluated for a series of pre-defined video events specified using a video event ontology.

show abstract

Automatic Video Interpretation: A Recognition Algorithm for Temporal Scenarios Based on Pre-compiled Scenario Models

Cited by 95 publications

References 2 publications

Monitoring Activities of Daily Living (ADLs) of Elderly Based on 3D Key Human Postures

Monitoring Activities of Daily Living (ADLs) of Elderly Based on 3D Key Human Postures

Audio-video event recognition system for public transport security

Automated Scene Understanding for Airport Aprons

Contact Info

Product

Resources

About