Labeling Images by Integrating Sparse Multiple Distance Learning and Semantic Context Modeling

Abstract

Abstract. Recent progress on Automatic Image Annotation (AIA) is achieved by either exploiting low level visual features or high level semantic context. Integrating these two paradigms to further leverage the performance of AIA is promising. However, very few previous works have studied this issue in a unified framework. In this paper, we propose a unified model based on Conditional Random Fields (CRF), which establishes tight interaction between visual features and semantic context. In particular, Kernelized Logistic Regression (KLR) with multiple visual distance learning is embedded into the CRF framework. We introduce L1 and L2 regularization terms into the unified learning process for the distance learning and the parameters penalty respectively. The experiments are conducted on two benchmarks: Corel and TRECVID-2005 data sets for evaluation. The experimental results show that, compared with the state-of-the-art methods, the unified model achieves significant improvement on annotation performance and shows more robustness with increasing number of various visual features.

Manifold Kernel Metric Learning for Larger-Scale Image Annotation

“…we compare our M KML algorithm to linear DML algorithms and kernel extension, i.e., Discriminative component analysis(DCA) and Kernel DCA(KDCA) [20], local fisher discriminant analysis(LFDA) and Kernel LFDA(KLFDA) [11], and several state-of-the-art image annotation models including: (1) Two versions of the TagProp method [8], using either rank based weights (TP-R) or distance-based weights (TP-D), (2) Tag Relevance (tRel) [9] based on the idea of neighbor voting, (3) 1-vs-1 SVM classification, using either linear (SVML) or RBF kernel (SVMK) classifiers. (4) RKML methods and their two versions extensions of RKML [22], which are RKMLH and RLML methods.…”

“…Since the big gap between image features and image semantics is always present [12], an appropriate distance metrics is very importance, and even determine the final performance. The early studies mainly focus on linear distance metrics, and they have been developed to learn a linear DML from pairwise constraints [13], and some of them are designed exclusively for image annotation [11], [14], [15]. However, images always have a nonlinear relationship and multi-modal patterns [16]- [19].…”

Tagging Personal Photos with Transfer Deep Learning

“…To address the problem of large-scale annotation of Web images, Visual synset applies multi-class one-vs-all linear Support Vector Machine models, which are learned from the automatically generated visual synsets of a large collection of Web images [27]. In [13], Ji et al exploit both low-level visual features and high-level semantic context into a unified conditional random fields (CRF) model for solving the image tagging problem, which achieves significant improvement and more robustness results on two benchmarks. Although the model-based approach can achieve good performance, it may suffer from a limited vocabulary and less scalability on largescale datasets.…”

Large-Scale Image Annotation by Efficient and Robust Kernel Metric Learning

“…A sparse coding scheme is proposed in [11] to facilitate label propagation. Conditional Random Field model is adopted in [17] to capture the spatial correlation between annotations of neighboring images.…”

“…Among various approaches developed for automatic image annotation, search based approaches have been proved to be quite effective, particularly for large image datasets with many keywords [12,17,21,29]. Their key idea is to annotate a test image I with the common tags shared by the subset of training images that are visually similar to I.…”