A model of multimedia information retrieval

Abstract

Abstract. Research on multimedia information retrieval (MIR) has recently witnessed a booming interest. A prominent feature of this research trend is its simultaneous but independent materialization within several fields of computer science. The resulting richness of paradigms, methods and systems may, on the long run, result in a fragmentation of efforts and slow down progress. The primary goal of this study is to promote an integration of methods and techniques for MIR by contributing a conceptual model that encompasses in a unified and coherent perspective the many efforts that are being produced under the label of MIR. The model offers a retrieval capability that spans two media, text and images, but also several dimensions: form, content and structure. In this way, it reconciles similarity-based methods with semantics-based ones, providing the guidelines for the design of systems that are able to provide a generalized multimedia retrieval service, in which the existing forms of retrieval not only coexist, but can be combined in any desired manner. The model is formulated in terms of a fuzzy description logic, which plays a twofold role: (1) it directly models semantics-based retrieval, and (2) it offers an ideal framework for the integration of the multimedia and multidimensional aspects of retrieval mentioned above. The model also accounts for relevance feedback in both text and image retrieval, integrating known techniques for taking into account user judgments. The implementation of the model is addressed by presenting a decomposition technique that reduces query evaluation to the processing of simpler requests, each of which can be solved by means of widely known methods for text and image retrieval, and semantic processing. A prototype for multidimensional image retrieval is presented that shows this decomposition technique at work in a significant case.

Foundations of a Multidimensional Query Language for Digital Libraries

“…The idea is to partition the language presented here into a set of sub-languages, each addressing a different dimension of documents, and use the general framework of logic for combining these sub-languages. In this respect, the inclusion of content-based retrieval in the language presents no conceptual problem, as a logical framework supporting this kind of functionality has already been developed [14]. An architecture for implementing the language is described in [9].…”

“…For simplicity, our model will only consider contents of type text, even though the extension to other types is possible [14]. As far as structure is concerned, we consider ordered sets and sequences.…”

Fuzzy Methods on the Web: A Critical Discussion

“…Most work on fuzzy description logics has been theoretical, developing more expressive formalisms, or more scalable reasoning mechanisms [13,75]. One notable application of fuzzy description logics is in multimedia retrieval [65], where fuzzy description logic are used to encode both semantic annotations of multimedia documents and the result of e.g. image processing analyses.…”

A Fuzzy Extension to the OWL 2 RL Ontology Language

“…For example, in our hypothetical scenario one cannot capture the fact that doctor "a" is better than doctor "b" who is still quite good though. Managing fuzzy knowledge is of great importance in many applications, like multimedia processing [5,6], decision making [7], negotiation [8], and more. For these reasons many fuzzy extensions to OWL and DLs have been proposed [9,10,11,12,13,14,15,16,17].…”