PicSOM-self-organizing image retrieval with MPEG-7 content descriptors

Abstract

Development of content-based image retrieval (CBIR) techniques has suffered from the lack of standardized ways for describing visual image content. Luckily, the MPEG-7 international standard is now emerging as both a general framework for content description and a collection of specific agreed-upon content descriptors. We have developed a neural, self-organizing technique for CBIR. Our system is named PicSOM and it is based on pictorial examples and relevance feedback (RF). The name stems from "picture" and the self-organizing map (SOM). The PicSOM system is implemented by using tree structured SOMs. In this paper, we apply the visual content descriptors provided by MPEG-7 in the PicSOM system and compare our own image indexing technique with a reference system based on vector quantization (VQ). The results of our experiments show that the MPEG-7-defined content descriptors can be used as such in the PicSOM system even though Euclidean distance calculation, inherently used in the PicSOM system, is not optimal for all of them. Also, the results indicate that the PicSOM technique is a bit slower than the reference system in starting to find relevant images. However, when the strong RF mechanism of PicSOM begins to function, its retrieval precision exceeds that of the reference system.

Improving Automatic Video Retrieval with Semantic Concept Detection

“…Our annual submissions have been, however, somewhat different each year due to modifications and additions to our PicSOM [12] retrieval system framework, to the used features and algorithms, etc. For brevity, only a general overview of the experiments and the used settings is provided in this paper.…”

“…More detailed descriptions can be found in our annual TRECVID workshop papers [13,14,15]. In all experiments, we combine content-based retrieval based on the topic-wise image and video examples using our standard SOM-based retrieval algorithm [12], concept-based retrieval with concept detectors trained as described in Section 2.1, and text search (c.f. Fig.…”

Computational Intelligence in Multimedia Processing: Foundation and Trends

“…Their initial results show a promising increase in retrieval performance. PicSOM [45] content-based information retrieval (CBIR) system was used with video data and semantic classes from the NIST TRECVID 20051 evaluation set. The TRECVID set contains TV broadcasts in different languages and textual data acquired by using automatic speech recognition software and machine translation where appropriate.…”

Focusing Keywords to Automatically Extracted Image Segments Using Self-Organising Maps

“…The system is implemented inside the PicSOM 1 CBIR software framework [27,28,29]. As an input, the system takes two sets of images: a set of images annotated with a certain keyword (positive examples), and a set of auxiliary background images (negative examples) As the result of the processing the system produces a segmentation of the positive example images and ranks the segments according to their relevance to the keyword, i.e.…”

“…These features include a set of MPEG-7 content descriptors [22,29] and additionally some non-standard descriptors for colour, shape and texture.…”