A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image Retrieval

Abstract

Abstract-Similarity measurement is a critical component in content-based image retrieval systems, and learning a good distance metric can significantly improve retrieval performance. However, despite extensive study, there are several major shortcomings with the existing approaches for distance metric learning that can significantly affect their application to medical image retrieval. In particular, "similarity" can mean very different things in image retrieval: resemblance in visual appearance (e.g., two images that look like one another) or similarity in semantic annotation (e.g., two images of tumors that look quite different yet are both malignant). Current approaches for distance metric learning typically address only one goal without consideration of the other. This is problematic for medical image retrieval where the goal is to assist doctors in decision making. In these applications, given a query image, the goal is to retrieve similar images from a reference library whose semantic annotations could provide the medical professional with greater insight into the possible interpretations of the query image. If the system were to retrieve images that did not look like the query, then users would be less likely to trust the system; on the other hand, retrieving images that appear superficially similar to the query but are semantically unrelated is undesirable because that could lead users toward an incorrect diagnosis. Hence, learning a distance metric that preserves both visual resemblance and semantic similarity is important. We emphasize that, although our study is focused on medical image retrieval, the problem addressed in this work is critical to many image retrieval systems. We present a boosting framework for distance metric learning that aims to preserve both visual and semantic similarities. The boosting framework first learns a binary representation using side information, in the form of labeled pairs, and then computes the distance as a weighted Hamming distance using the learned binary representation. A boosting algorithm is presented to efficiently learn the distance function. We evaluate the proposed algorithm on a mammographic image reference library with an Interactive Search-Assisted Decision Support (ISADS) system and on the medical image data set from ImageCLEF. Our results show that the boosting framework compares favorably to state-of-the-art approaches for distance metric learning in retrieval accuracy, with much lower computational cost. Additional evaluation with the COREL collection shows that our algorithm works well for regular image data sets.

An agent-based algorithm exploiting multiple local dissimilarities for clusters mining and knowledge discovery

“…This procedure is strongly related to the problem of the metric-learning [44,48,49,50,14], which is the task of determining the optimal parameters of a given metric distance. However, in our case we make no a-priori assumptions on the adopted distance (which we call dissimilarity measure in our study).…”

“…Metric learning [44,48,49,50,14] is an important subfield of pattern recognition. Techniques in this field deal with the problem of learning an optimal setting of the parameters characterizing the particular dissimilarity for the problem at hand -usually it is assumed to be a metric distance.…”

Multiview Locally Linear Embedding for Effective Medical Image Retrieval

“…The IRMA medical image data set is a popular benchmark database used to evaluate CBMIR [6], [12], [42], [43]. The new version of the IRMA medical image data set [40] contains 193 categories with a total of 12,677 fully annotated gray value radiographs in a training set.…”

Content-Based Image Retrieval Using Spatial Layout Information in Brain Tumor T1-Weighted Contrast-Enhanced MR Images

“…Given that low-level visual features, such as color (intensity), texture [12]–[17], and shape [18], [19], are not adequately discriminative to describe high-level semantic concepts [20], additional distinctive features are highly desirable [6], [7]. Second, learned distance metrics, which are capable of reducing the semantic gap between visual features and semantic concepts [21], have been investigated intensively and widely used for CBIR [6], [22], [23].…”