A Simple Exponential Family Framework for Zero-Shot Learning

Verma, Vinay Kumar; Rai, Piyush

doi:10.1007/978-3-319-71246-8_48

Cited by 163 publications

(129 citation statements)

References 25 publications

Supporting

Mentioning

127

Contrasting

Order By: Relevance

“…Observe that the proposed methods, Ours(S) and Ours(Π) consistently outperforms state-of-the-art methods in the GZSL setting. Specifically, the harmonic mean of the accuracy for seen (tr) and unseen (ts) classes with Ours(S) and [21,12,19,28]) only perform well on the seen classes and obtain close-to-zero accuracy on unseen classes, we are able to classify both seen and unseen improving upon existing works in the GZSL setting.…”

Section: Generalized Zero Shot Learning Evaluationmentioning

confidence: 79%

Generalized Zero-Shot Recognition Based on Visually Semantic Embedding

Zhu

Wang

Saligrama

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We propose a novel Generalized Zero-Shot learning (GZSL) method that is agnostic to both unseen images and unseen semantic vectors during training. Prior works in this context propose to map high-dimensional visual features to the semantic domain, which we believe contributes to the semantic gap. To bridge the gap, we propose a novel low-dimensional embedding of visual instances that is "visually semantic." Analogous to semantic data that quantifies the existence of an attribute in the presented instance, components of our visual embedding quantifies existence of a prototypical part-type in the presented instance. In parallel, as a thought experiment, we quantify the impact of noisy semantic data by utilizing a novel visual oracle to visually supervise a learner. These factors, namely semantic noise, visual-semantic gap and label noise lead us to propose a new graphical model for inference with pairwise interactions between label, semantic data, and inputs. We tabulate results on a number of benchmark datasets demonstrating significant improvement in accuracy over state-of-art under both semantic and visual supervision.

show abstract

Section: Generalized Zero Shot Learning Evaluationmentioning

confidence: 79%

Generalized Zero-Shot Recognition Based on Visually Semantic Embedding

Zhu

Wang

Saligrama

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…[26,16] exploit semantic manifold learning. GFZSL [52] treats unknown labels of unseen class images as latent variables and applies Expectation-Maximization (EM). As the prediction is biased to seen classes in GZSL, UE [51] maximizes the probability of predicting unlabeled images as unseen classes.…”

Section: Related Workmentioning

confidence: 99%

F-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning

Xian

Sharma

Schiele³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

487

358

View full text Add to dashboard Cite

When labeled training data is scarce, a promising data augmentation approach is to generate visual features of unknown classes using their attributes. To learn the class conditional distribution of CNN features, these models rely on pairs of image features and class attributes. Hence, they can not make use of the abundance of unlabeled data samples. In this paper, we tackle any-shot learning problems i.e. zero-shot and few-shot, in a unified feature generating framework that operates in both inductive and transductive learning settings. We develop a conditional generative model that combines the strength of VAE and GANs and in addition, via an unconditional discriminator, learns the marginal feature distribution of unlabeled images. We empirically show that our model learns highly discriminative CNN features for five datasets, i.e. CUB, SUN, AWA and ImageNet, and establish a new state-of-the-art in any-shot learning, i.e. inductive and transductive (generalized) zeroand few-shot learning settings. We also demonstrate that our learned features are interpretable: we visualize them by inverting them back to the pixel space and we explain them by generating textual arguments of why they are associated with a certain label.

show abstract

“…It nevertheless uses a single projection matrix to project visual features into the semantic space. More recently, the authors of [30] proposed to learn generative models to predict data distribution of seen and unseen classes from their attribute vectors, and used unlabeled test data to refine the distribution parameters of target classes. The work in [28] trains an end-to-end network that optimizes the loss on both seen class data and unseen test data, by minimizing the Quasi-Fully Supervised Learning loss, which uses target class data to reduce seen/unseen bias of the model during training.…”

Section: Transductive Zero-shot Learningmentioning

confidence: 99%

Progressive Ensemble Networks for Zero-Shot Recognition

Guo

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Despite the advancement of supervised image recognition algorithms, their dependence on the availability of labeled data and the rapid expansion of image categories raise the significant challenge of zero-shot learning. Zero-shot learning (ZSL) aims to transfer knowledge from labeled classes into unlabeled classes to reduce human labeling effort. In this paper, we propose a novel progressive ensemble network model with multiple projected label embeddings to address zero-shot image recognition. The ensemble network is built by learning multiple image classification functions with a shared feature extraction network but different label embedding representations, which enhance the diversity of the classifiers and facilitate information transfer to unlabeled classes. A progressive training framework is then deployed to gradually label the most confident images in each unlabeled class with predicted pseudo-labels and update the ensemble network with the training data augmented by the pseudo-labels. The proposed model performs training on both labeled and unlabeled data. It can naturally bridge the domain shift problem in visual appearances and be extended to the generalized zero-shot learning scenario. We conduct experiments on multiple ZSL datasets and the empirical results demonstrate the efficacy of the proposed model.

show abstract

A Simple Exponential Family Framework for Zero-Shot Learning

Cited by 163 publications

References 25 publications

Generalized Zero-Shot Recognition Based on Visually Semantic Embedding

Generalized Zero-Shot Recognition Based on Visually Semantic Embedding

F-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning

Progressive Ensemble Networks for Zero-Shot Recognition

Contact Info

Product

Resources

About