Zero-Shot Image Classification Based on a Learnable Deep Metric

Abstract

The supervised model based on deep learning has made great achievements in the field of image classification after training with a large number of labeled samples. However, there are many categories without or only with a few labeled training samples in practice, and some categories even have no training samples at all. The proposed zero-shot learning greatly reduces the dependence on labeled training samples for image classification models. Nevertheless, there are limitations in learning the similarity of visual features and semantic features with a predefined fixed metric (e.g., as Euclidean distance), as well as the problem of semantic gap in the mapping process. To address these problems, a new zero-shot image classification method based on an end-to-end learnable deep metric is proposed in this paper. First, the common space embedding is adopted to map the visual features and semantic features into a common space. Second, an end-to-end learnable deep metric, that is, the relation network is utilized to learn the similarity of visual features and semantic features. Finally, the invisible images are classified, according to the similarity score. Extensive experiments are carried out on four datasets and the results indicate the effectiveness of the proposed method.

A review of few-shot image recognition using semantic information

“…Both types of information are projected onto the MKE space to infer similarities between seen and unseen classes. A zero-shot image classification method based on a learnable deep metric (ZIC-LDM) [35] was proposed to learn a common space so that image and semantic features can be mapped onto this space to help with the semantic gap problem. In several papers [36][37][38][39][40][41], auto-encoders were used to map visual and semantic information onto the latent space.…”

“…The present study focused on semantic-based few-shot image recognition, so the following recommendations for researchers are mainly on this topic. The most critical issue for semantic-based FSL is the domain shift problem [26,35]. Visual and semantic features have different properties and cannot be combined directly.…”

GMRNet: A Novel Geometric Mean Relation Network for Few-Shot Very Similar Object Classification

“…The goal of this module is to calculate the degree of similarity between samples and to generate a correlation result [28]. Dense, Activation Relu, and Dropout layers were used in this module.…”