Learning a weighted semantic manifold for content-based image retrieval

Abstract

We propose a novel weighted semantic manifold ranking system for content-based image retrieval. This manifold builds a more accurate intrinsic structure for the proper image space by combining visual and semantic relevance relations. Specifically, we apply the learning mechanism to capture users' semantic concepts in clusters and extract high-level semantic features for each database image. We then incorporate the reliability score, the fuzzy membership, and the composite low-level and high-level relation into the traditional affinity matrix to construct a weighted semantic manifold structure. We finally create an asymmetric relevance vector to propagate positive and negative labels via the proposed manifold structure to images with high similarities. Extensive experiments demonstrate our system outperforms other manifold systems and learning systems in the context of both correct and erroneous feedback.

Block-based long-term content-based image retrieval using multiple features

“…This large matrix prevents some CBIR systems from searching a large-scale image database. For example, manifold ranking systems [8,9] require several coexisting large matrices to propagate learned labels. They cannot run on a computer if the number of images is over 10,000 due to the large consumption of the memory space.…”

“…They then derive the semantic relevance among images and combine the semantic similarity with the visual similarity to find top images that are similar to the query image. Chang et al [8,9] use RF to group similar images into semantic clusters and further use semantic clusters based information to construct a weighted manifold structure in two ways to propagate the ranking scores of labeled images. Although these learning techniques achieve impressive retrieval results, they usually require a large matrix to store historical feedback information to learn the relationships among all images.…”

“…However, most existing RF techniques use shortterm (intra-query) learning to handle query formulation in a single retrieval session. Recently, long-term (inter-query) learning [2][3][4][5][6][7][8][9] extends short-term learning to derive semantic meaning of database images by studying the feedback history collected from multiple users in different retrieval sessions. Long-term learning has mainly been studied in two ways.…”

Complementary relevance feedback-based content-based image retrieval

“…The merging threshold for creating semantic clusters is set to be 0.5, the parameter σ (the overall variance of image features) for computing the weight in the affinity matrix is set to be 0.05, the convergence rate of the affinity matrix α is set to be 0.99, and the parameter γ in the RBF kernel is set to be 0.5. & Chang's long-term weighted semantic manifold ranking system (i.e., Semantic manifold) [3]: This system builds upon SC-based manifold to include the fuzzy membership and the high-level semantic similarity score in the affinity matrix to construct a semantic manifold ranking system. All the fixed parameters are the same as the ones used in [2].…”

“…They then incorporate the reliability scores of each database image into the affinity matrix to construct a weighted manifold structure to discriminately propagate the ranking scores of labeled images. They further expand this weighted manifold structure by incorporating the reliability scores, the fuzzy membership, and the high-level semantic similarity score into the affinity matrix to construct a semantic manifold structure [3]. These learning techniques reduce the semantic gap and achieve impressive retrieval results.…”