Attributes2Classname: A Discriminative Model for Attribute-Based Unsupervised Zero-Shot Learning

Abstract

We propose a novel approach for unsupervised zero-shot learning (ZSL) of classes based on their names. Most existing unsupervised ZSL methods aim to learn a model for directly comparing image features and class names. However, this proves to be a difficult task due to dominance of non-visual semantics in underlying vector-space embeddings of class names. To address this issue, we discriminatively learn a word representation such that the similarities between class and combination of attribute names fall in line with the visual similarity. Contrary to the traditional zero-shot learning approaches that are built upon attribute presence, our approach bypasses the laborious attributeclass relation annotations for unseen classes. In addition, our proposed approach renders text-only training possible, hence, the training can be augmented without the need to collect additional image data. The experimental results show that our method yields state-of-the-art results for unsupervised ZSL in three benchmark datasets.

Learning Visually Consistent Label Embeddings for Zero-Shot Learning

“…Lampert et al [17,11] show that using attributes provide convenient and cost effective knowledge transfer between seen and unseen classes. Demirel et al [10] use attribute information to learn visually more meaningful word representations. Unlike most other attribute based approaches, their method does not require the human supervised attribute-class relations at test time.…”

“…For example, semantically similar words, such "wolf" and "bear" are not particularly close in visual domain, whereas visually consistent words such as "mole" and "mouse" can be far apart in semantic word domain. In order to account for such differences, [10] propose to learn a transformation on the word vectors that allows ZSL by comparing the pooled embeddings of attribute names and class names. Below we provide only a brief summary of the image-based training formulation of this approach, a more through explanation can be found in [10].…”

“…In order to account for such differences, [10] propose to learn a transformation on the word vectors that allows ZSL by comparing the pooled embeddings of attribute names and class names. Below we provide only a brief summary of the image-based training formulation of this approach, a more through explanation can be found in [10].…”

“…For AwA and aPaY datasets, we utilize the CNN-M2K features [9], where images are resized to 256x256 and mean image subtraction is applied. Outputs of the last hidden layer are extracted for image representation, as also described in [10].…”

“…[4,5,6]) or attribute-based class specifications (e.g. [1,7,8,9,4,6,10]). Overall, the performance of a ZSL method heavily depends on the prior information as it is the primary factor determining the limits of cross-class knowledge sharing and transfer.…”