Recognizing Complex Events Using Large Margin Joint Low-Level Event Model

Abstract

Abstract. In this paper we address the challenging problem of complex event recognition by using low-level events. In this problem, each complex event is captured by a long video in which several low-level events happen. The dataset contains several videos and due to the large number of videos and complexity of the events, the available annotation for the low-level events is very noisy which makes the detection task even more challenging. To tackle these problems we model the joint relationship between the low-level events in a graph where we consider a node for each low-level event and whenever there is a correlation between two low-level events the graph has an edge between the corresponding nodes. In addition, for decreasing the effect of weak and/or irrelevant low-level event detectors we consider the presence/absence of low-level events as hidden variables and learn a discriminative model by using latent SVM formulation. Using our learned model for the complex event recognition, we can also apply it for improving the detection of the low-level events in video clips which enables us to discover a conceptual description of the video. Thus our model can do complex event recognition and explain a video in terms of low-level events in a single framework. We have evaluated our proposed method over the most challenging multimedia event detection dataset. The experimental results reveals that the proposed method performs well compared to the baseline method. Further, our results of conceptual description of video shows that our model is learned quite well to handle the noisy annotation and surpass the low-level event detectors which are directly trained on the raw features.

Complex event recognition by latent temporal models of concepts

“…Both the concept detectors and the HCRF have been trained on the training set alone, and the test videos have been used blindly for testing without any further adjustment of the parameters. As evaluation metric, we have adopted the average precision which is an average of the precision at various levels of recall (equivalent to the area under the precision-recall curve) [8,6]. A total number of 93 concepts were annotated over a portion of the training data.…”

“…This result is very important as it shows that, despite their complex nature, many events can be well characterized by features of low-level semantics [6,7,8]. However, hierarchical approaches have also become increasingly popular in recent years where more general "concepts" are first identified and then used as atoms for the characterization and recognition of complex events [9,10,6,7].…”

“…Loui et al in [10] collected a benchmark dataset containing 25 general concepts: however, they are based on static images, not videos. Concepts have also been deployed in the form of attributes [11], which can be considered as concepts with small granularity [6]. [7] used deep learning to find data-driven concepts.…”

Discovering Video Clusters from Visual Features and Noisy Tags

“…Video recognition has been an active research area in computer vision. For example, Izadinia and Shah [12] recognized complex video events (e.g. "parade", "landing fish") from low-level event tags (e.g.…”

Clauselets: Leveraging Temporally Related Actions for Video Event Analysis

“…Izadinia and Shah [9] model the joint relationship between two actions for recognizing high-level event. While pairs of actions capture more information than single actions alone, valuable information from higher order interactions remains unused.…”