Online Multi-Modal Distance Metric Learning with Application to Image Retrieval

Wu, Pengcheng; Hoi, Steven C. H.; Zhao, Peilin; Miao, Chunyan; Liu, Zhiyong

doi:10.1109/tkde.2015.2477296

Cited by 76 publications

(39 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, there has been an increasing interest in the information retrieval and multimedia computing communities to study smart image retrieval techniques. In particular, techniques for content-based image retrieval (CBIR) [1], [2], where only visual image is used as query, are gaining in importance due to a wide range of promising applications.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Visual Hashing with Semantic Assistant for Content-Based Image Retrieval

Zhu

Shen

Xie

et al. 2017

IEEE Trans. Knowl. Data Eng.

175

View full text Add to dashboard Cite

As an emerging technology to support scalable content-based image retrieval (CBIR), hashing has recently received great attention and became a very active research domain. In this study, we propose a novel unsupervised visual hashing approach called semantic-assisted visual hashing (SAVH). Distinguished from semi-supervised and supervised visual hashing, its core idea is to effectively extract the rich semantics latently embedded in auxiliary texts of images to boost the effectiveness of visual hashing without any explicit semantic labels. To achieve the target, a unified unsupervised framework is developed to learn hash codes by simultaneously preserving visual similarities of images, integrating the semantic assistance from auxiliary texts on modeling high-order relationships of inter-images, and characterizing the correlations between images and shared topics. Our performance study on three publicly available image collections: Wiki, MIR Flickr, and NUS-WIDE indicates that SAVH can achieve superior performance over several state-of-the-art techniques.

show abstract

Section: Introductionmentioning

confidence: 99%

Unsupervised Visual Hashing with Semantic Assistant for Content-Based Image Retrieval

Zhu

Shen

Xie

et al. 2017

IEEE Trans. Knowl. Data Eng.

175

View full text Add to dashboard Cite

show abstract

“…SoDA is an online learning methods. In literatures, online learning [2], [6], [12], [40], [11] is known as a light and rapid means to process streaming data or large-scale datasets, and it has been widely exploited in many real-world tasks such as Face Recognition [14], [36], Images Retrieval [21], [42] and Object Tracking [19], [18]. It enables learning a up-to-date model based on streaming data.…”

Section: Related Workmentioning

confidence: 99%

One-pass person re-identification by sketch online discriminant analysis

Zhong

Zheng

2019

Pattern Recognition

View full text Add to dashboard Cite

Person re-identification (re-id) is to match people across disjoint camera views in a multi-camera system, and re-id has been an important technology applied in smart city in recent years. However, the majority of existing person re-id methods are not designed for processing sequential data in an online way. This ignores the real-world scenario that person images detected from multi-cameras system are coming sequentially. While there is a few work on discussing online re-id, most of them require considerable storage of all passed data samples that have been ever observed, and this could be unrealistic for processing data from a large camera network. In this work, we present an onepass person re-id model that adapts the re-id model based on each newly observed data and no passed data are directly used for each update. More specifically, we develop an Sketch online Discriminant Analysis (SoDA) by embedding sketch processing into Fisher discriminant analysis (FDA). SoDA can efficiently keep the main data variations of all passed samples in a low rank matrix when processing sequential data samples, and estimate the approximate within-class variance (i.e. within-class covariance matrix) from the sketch data information. We provide theoretical analysis on the effect of the estimated approximate within-class covariance matrix. In particular, we derive upper and lower bounds on the Fisher discriminant score (i.e. the quotient between between-class variation and within-class variation after feature transformation) in order to investigate how the optimal feature transformation learned by SoDA sequentially approximates the offline FDA that is learned on all observed data. Extensive experimental results have shown the effectiveness of our SoDA and empirically support our theoretical analysis.

show abstract

“…different distance metrics in different feature spaces [38] . To facilitate tasks such as inter-modal label transfer and zero-shot learning, multi-modal models are developed to formulate the relations between text and image features [31].…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Attention driven multi-modal similarity learning

Gao

Goulermas

et al. 2018

Information Sciences

View full text Add to dashboard Cite

To learn a function for measuring similarity or relevance between objects is an important machine learning task, referred to as similarity learning. Conventional methods are usually insufficient for processing complex patterns, while more sophisticated methods produce results supported by parameters and mathematical operations that are hard to interpret. To improve both model robustness and interpretability, we propose a novel attention driven multi-modal algorithm, which learns a distributed similarity score over different relation modalities and develops an interaction-oriented dynamic attention mechanism to selectively focus on salient patches of objects of interest. Neural networks are used to generate a set of high-level representation vectors for both the entire object and its segmented patches. Multi-view local neighboring structures between objects are encoded in the high-level object representation through an unsupervised pre-training procedure. By initializing the relation embeddings with object cluster centers, each relation modality can be reasonably interpreted as a semantic topic. A layer-wise training scheme based on a mixture of unsupervised and supervised training is proposed to improve generalization. The effectiveness of the proposed method and its superior performance compared against state-of-the-art algorithms are demonstrated through evaluations based on different image retrieval tasks.

show abstract

Online Multi-Modal Distance Metric Learning with Application to Image Retrieval

Cited by 76 publications

References 38 publications

Unsupervised Visual Hashing with Semantic Assistant for Content-Based Image Retrieval

Unsupervised Visual Hashing with Semantic Assistant for Content-Based Image Retrieval

One-pass person re-identification by sketch online discriminant analysis

Attention driven multi-modal similarity learning

Contact Info

Product

Resources

About