A semantic-based probabilistic approach for real-time video event recognition

Abstract

El acceso a la versión del editor puede requerir la suscripción del recurso Access to the published version may require subscription This paper presents an approach for real-time video event recognition that combines the accuracy and descriptive capabilities of, respectively, probabilistic and semantic approaches. Based on a state-of-art knowledge representation, we de ne a methodology for building recognition strategies from event descriptions that consider the uncertainty of the low-level analysis. Then, we e ciently organize such strategies for performing the recognition according to the temporal characteristics of events. In particular, we use Bayesian Networks and probabilistically-extended Petri Nets for recognizing, respectively, simple and complex events. For demonstrating the proposed approach, a framework has been implemented for recognizing human-object interactions in the video monitoring domain. The experimental results show that our approach improves the event recognition performance as compared to the widely used deterministic approach.Keywords: Video event detection, Semantic video analysis, Bayes Network, Petri Net, Low-level uncertaintyThe recognition of human-related events has recently become a relevant research area motivated by the variety of promising applications such as video surveillance, human-computer interaction and content-based indexing. Moreover, this interest can be also explained by the maturity of the employed low-level tools. Nevertheless, it still presents many challenges such as the uncertainty of the low-level tools (e.g., object detection and tracking), the limited availability of training data, the similar appearance of di erent events and the modeling of complex relations.Many approaches have been proposed for event recognition which can be roughly classi ed into semantic and probabilistic. Semantic (or deterministic) approaches are based on de ning rules to model the events [1]. However, current approaches only describe a small portion of semantics (e.g., $ This work has been partially supported by the Spanish Administration agency CDTI (CENIT-VISION 2007-1007 On the other hand, the probabilistic approaches have shown a superior performance as compared to the semantic ones [4]. They accurately learn event models from training data achieving high precision within a domain and allowing an intrinsic uncertainty handling. However, they are not able to model complex relations and their usage is limited for di erent, albeit related, domains. In this situation, a combination of both approaches would be desirable for solving these limitations. Although this combination is gaining attention in the recent years, current approaches are limited to the de nition of simple events [5], the assumption of accurate low-level analysis [6] and the use of domain-dependent recognition strategies [7]. Thus, their extension to generic recognition of complex events considering low-level uncertainty is not a straightforward task. This paper addresses the above-mentioned limitations by introducin...

Evidence Reasoning for Event Inference in Smart Transport Video Surveillance

“…There are roughly two different groups of approaches for event recognition developed up to date: deterministic and probabilistic [16]. Of the deterministic approaches, syntactic techniques such as context-free grammar [14], description methodologies such as Petri Nets [8], and logic-based approaches, such as Allen's temporal predicates [1], have been used to model events in real applications.…”

“…Of the deterministic approaches, syntactic techniques such as context-free grammar [14], description methodologies such as Petri Nets [8], and logic-based approaches, such as Allen's temporal predicates [1], have been used to model events in real applications. These approaches can be used to describe the semantics of events, but lack an appropriate recognition step and do not consider uncertainty related issues [16]. For the probabilistic approaches [7][15], a probabilistic model is constructed from training data.…”

Skin detection by dual maximization of detectors agreement for video monitoring

“…Recently, recognition of human activities in video has become a relevant topic where the detection and tracking of body parts, via skin detection, plays a key role [1] [2]. However, such detection faces many challenges related to the scenario (illumination changes and backgrounds with skin-like surfaces), the field of view (medium-small skin areas) and the limited availability of training data, which decrease the performance of traditional skin detection approaches for this recognition task.…”

Hierarchical Task Network planning with common-sense reasoning for multiple-people behaviour analysis

“…While probabilistic approaches can overcome 180 the limitations of ontological languages (SanMiguel & Martinez, 2012) (Cilla et al, 2012), they require precise quantification of the uncertainty associate to every sensor and situation. In real world scenarios, such as public-transport platforms and other highly dynamic environments, deriving accurate probabilities for human activities is fraught with difficulties, since this values tend to vary 185 over time and with the observed conditions and number of subjects (Kuipers, 1994).…”