Ontology-enriched semantic space for video search

Abstract

Multimedia-based ontology construction and reasoning have recently been recognized as two important issues in video search, particularly for bridging semantic gap. The lack of coincidence between low-level features and user expectation makes concept-based ontology reasoning an attractive midlevel framework for interpreting high-level semantics. In this paper, we propose a novel model, namely ontology-enriched semantic space (OSS), to provide a computable platform for modeling and reasoning concepts in a linear space. OSS enlightens the possibility of answering conceptual questions such as a high coverage of semantic space with minimal set of concepts, and the set of concepts to be developed for video search. More importantly, the query-to-concept mapping can be more reasonably conducted by guaranteeing the uniform and consistent comparison of concept scores for video search. We explore OSS for several tasks including conceptbased video search, word sense disambiguation and multimodality fusion. Our empirical findings show that OSS is a feasible solution to timely issues such as the measurement of concept combination and query-concept dependent fusion.

Selection of Concept Detectors for Video Search by Ontology-Enriched Semantic Spaces

“…We propose two variants of the semantic space by considering orthogonality property of the space. The first space is named Ontologyenriched Semantic Space (OSS), originally presented in our recent work [9]. The second space is called Ontology-enriched Orthogonal Semantic Space (OS 2 ).…”

“…Constructing OSS OSS, originally proposed in [9], aims to make the semantic space in Eqn (7) as compact and complete as possible. Similar to the anchor selection approaches, OSS achieves the aim by identifying a subset of concepts in C appropriate to serve as the basis vectors of the semantic space.…”

“…This process is similar to OSS where the medoid of each cluster is picked to formed the new setV of m < n concepts. A reduced matrix R of m-by-m size is computed, and then decomposed via Eqn (9). The coordinate system of OS 2 is then estimated and represented with the eigenvectors of Eqn (9).…”

“…A reduced matrix R of m-by-m size is computed, and then decomposed via Eqn (9). The coordinate system of OS 2 is then estimated and represented with the eigenvectors of Eqn (9). The semantic space is spanned by m < n basis concepts, and thus is compact and relatively efficient when predicting the unseen concepts using Eqn (11).…”

“…Table I shows few examples of the 366 concept clusters and their medoids. The medoid concepts are selected as the basis vectors of OSS, while for OS 2 the medoids form the reduced vocabulary set V for spectral decomposition in Eqn (9). The next subsection will further investigate the impact of concept clusters to the search performance.…”