Structured natural‐language descriptions for semantic content retrieval of visual materials

Abstract

Keyword search of multimedia collections lacks precision and automatic parsing of unrestricted natural language annotations lacks accuracy. We propose a structure for natural language descriptions of the semantic content of visual materials that requires descriptions to be (modified) keywords, phrases, or simple sentences, with components that are grammatical relations common to many languages. This structure makes it easy to implement a collection's descriptions as a relational database, enabling efficient search via the application of well-developed database-indexing methods. Description components may be elements from external resources (thesaurus, ontology, database, or knowledge base). This provides a rich superstructure for the meaningful retrieval of images by their semantic contents.

Controlling time is power: An analysis of filmic metaphors in the Lotus advertisement «Shine»

“…In this seven-step method, we have to, first, describe the narrative discourse of the filmic texts by segmenting the advertisement in the smallets units of analysis, identify the modes, attach abstract meaning. Then, we need to structure the referential description (Tam and Leung 2001), find the incongruities and integrate them by comparisons, test if the comparisons are crossdomain, and test if the comparison is depicted as some kind indirect discourse about the topic. Finally, if the last steps are positive we will annotate the filmic text for metaphor.…”

Knowledge-Based Image Annotating

“…We consider IM as a set of Objects with particular characteristics, and a number of Relations between them. Following (Hollink et al, 2003) IMs are based on predicate information -triads (agent-processobject) and the settings (time, location and artist) (Tam et al, 2001). Using this method for our task, we would have two problems:…”

“…Description template that was proposed in (Tam et al, 2001) and used in (Schreiber et al, 2001), (Hollink et al, 2003) consisting of triads and settings is not possible for our corpus since our images are static, and they have usually no action at all. The only type of physical processes that are mentioned in the names of photographs are like this "Rafts sitting on the rocks at the village of Kurya".…”

Reseñas

“…It not only introduces the description of VISMIP's operational steps but also their theoretical justification. The method leads researchers to (i) understanding the general meaning of the picture by identifying its referential and abstract signification, its topic and also its message, (ii) structuring the referential description with Tam and Leung's (2001) VISMIP represents a relevant advance to multimodal and metaphor studies, since the existence of a tool with structural steps that do not leave any decisions towards the identification of metaphorical units to the personal intuition of researchers might lead, consequently, to a high level of consistent results among independent analysts.…”