Write a Classifier: Zero-Shot Learning Using Purely Textual Descriptions

Elhoseiny, Mohamed; Saleh, Babak; Elgammal, Ahmed

doi:10.1109/iccv.2013.321

Cited by 243 publications

(259 citation statements)

References 33 publications

(52 reference statements)

Supporting

Mentioning

259

Contrasting

Order By: Relevance

“…Finally, by modeling the label relations and ensuring consistency between visual predictions and semantic relations, our approach relates to work in transfer learning [31,30], zero-shot learning [28,12,25], and attribute-based recognition [1,36,33,14], especially those that use semantic knowledge to improve recognition [16,30] and those that propagate or borrow annotations between categories [26,22].…”

Section: Related Workmentioning

confidence: 99%

Large-Scale Object Classification Using Label Relation Graphs

Deng

Ding

Jia

et al. 2014

Lecture Notes in Computer Science

336

305

View full text Add to dashboard Cite

Abstract. In this paper we study how to perform object classification in a principled way that exploits the rich structure of real world labels. We develop a new model that allows encoding of flexible relations between labels. We introduce Hierarchy and Exclusion (HEX) graphs, a new formalism that captures semantic relations between any two labels applied to the same object: mutual exclusion, overlap and subsumption. We then provide rigorous theoretical analysis that illustrates properties of HEX graphs such as consistency, equivalence, and computational implications of the graph structure. Next, we propose a probabilistic classification model based on HEX graphs and show that it enjoys a number of desirable properties. Finally, we evaluate our method using a large-scale benchmark. Empirical results demonstrate that our model can significantly improve object classification by exploiting the label relations.

show abstract

Section: Related Workmentioning

confidence: 99%

Large-Scale Object Classification Using Label Relation Graphs

Deng

Ding

Jia

et al. 2014

Lecture Notes in Computer Science

336

305

View full text Add to dashboard Cite

show abstract

“…More recent studies take advantages of an embedding approach as middle layers between low-level features and class labels [4], [7], [8], [9], [10], [11]. Besides, some novel works study how to directly construct classifiers for unseen classes [12], [13], [14]. The latter stream focuses on how to effectively represent human knowledge that can generalise to novel classes, such as human-nameable attributes [2], [15], [16], [17], [18], word vectors [3], [19], textual descriptions [20], and class similarities [21], [22].…”

Section: Introductionmentioning

confidence: 99%

Zero-Shot Learning Using Synthesised Unseen Visual Data with Diffusion Regularisation

Long

Liu

Shen

et al. 2018

IEEE Trans. Pattern Anal. Mach. Intell.

103

View full text Add to dashboard Cite

Abstract-Sufficient training examples are the fundamental requirement for most of the learning tasks. However, collecting welllabelled training examples is costly. Inspired by Zero-shot Learning (ZSL) that can make use of visual attributes or natural language semantics as an intermediate level clue to associate low-level features with high-level classes, in a novel extension of this idea, we aim to synthesise training data for novel classes using only semantic attributes. Despite the simplicity of this idea, there are several challenges. Firstly, how to prevent the synthesised data from over-fitting to training classes? Secondly, how to guarantee the synthesised data is discriminative for ZSL tasks? Thirdly, we observe that only a few dimensions of the learnt features gain high variances whereas most of the remaining dimensions are not informative. Thus, the question is how to make the concentrated information diffuse to most of the dimensions of synthesised data. To address the above issues, we propose a novel embedding algorithm named Unseen Visual Data Synthesis (UVDS) that projects semantic features to the high-dimensional visual feature space. Two main techniques are introduced in our proposed algorithm. (1) We introduce a latent embedding space which aims to reconcile the structural difference between the visual and semantic spaces, meanwhile preserve the local structure. (2) We propose a novel Diffusion Regularisation (DR) that explicitly forces the variances to diffuse over most dimensions of the synthesised data. By an orthogonal rotation (more precisely, an orthogonal transformation), DR can remove the redundant correlated attributes and further alleviate the over-fitting problem. On four benchmark datasets, we demonstrate the benefit of using synthesised unseen data for zero-shot learning. Extensive experimental results suggest that our proposed approach significantly outperforms the state-of-the-art methods.

show abstract

“…Such issues make using attributes impractical. As a low-cost solution, text-based semantic features is proposed [32,10,37,26]. However, the textual description from the Internet can be noisy and not directly related to the visual appearance.…”

Section: Related Workmentioning

confidence: 99%

Describing Unseen Classes by Exemplars: Zero-Shot Learning Using Grouped Simile Ensemble

Long

Shao

2017

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Learning visual attributes is an effective approach for zero-shot recognition. However, existing methods are restricted to learning explicitly nameable attributes and cannot tell which attributes are more important to the recognition task. In this paper, we propose a unified framework named Grouped Simile Ensemble (GSE). We claim our contributions as follows. 1) We propose to substitute explicit attribute annotation by similes, which are more natural expressions that can describe complex unseen classes. Similes do not involve extra concepts of attributes, i.e. only exemplars of seen classes are needed. We provide an efficient scenario to annotate similes for two benchmark datasets, AwA and aPY. 2) We propose a graph-cut-based class clustering algorithm to effectively discover implicit attributes from the similes. 3) Our GSE can automatically find the most effective simile groups to make the prediction. On both datasets, extensive experimental results manifest that our approach can significantly improve the performance over the state-of-the-art methods.

show abstract

Write a Classifier: Zero-Shot Learning Using Purely Textual Descriptions

Cited by 243 publications

References 33 publications

Large-Scale Object Classification Using Label Relation Graphs

Large-Scale Object Classification Using Label Relation Graphs

Zero-Shot Learning Using Synthesised Unseen Visual Data with Diffusion Regularisation

Describing Unseen Classes by Exemplars: Zero-Shot Learning Using Grouped Simile Ensemble

Contact Info

Product

Resources

About