Adaptive Confidence Smoothing for Generalized Zero-Shot Learning

Abstract

Generalized zero-shot learning (GZSL) is the problem of learning a classifier where some classes have samples and others are learned from side information, like semantic attributes or text description, in a zero-shot learning fashion (ZSL). Training a single model that operates in these two regimes simultaneously is challenging. Here we describe a probabilistic approach that breaks the model into three modular components, and then combines them in a consistent way. Specifically, our model consists of three classifiers: A "gating" model that makes soft decisions if a sample is from a "seen" class, and two experts: a ZSL expert, and an expert model for seen classes. We address two main difficulties in this approach: How to provide an accurate estimate of the gating probability without any training samples for unseen classes; and how to use expert predictions when it observes samples outside of its domain.The key insight to our approach is to pass information between the three models to improve each one's accuracy, while maintaining the modular structure. We test our approach, adaptive confidence smoothing (COSMO ), on four standard GZSL benchmark datasets and find that it largely outperforms state-of-the-art GZSL models. COSMO is also the first model that closes the gap and surpasses the performance of generative models for GZSL, even-though it is a light-weight model that is much easier to train and tune. Notably, COSMO offers a new view for developing zeroshot models. Thanks to COSMO's modular structure, instead of trying to perform well both on seen and on unseen classes, models can focus on accurate classification of unseen classes, and later consider seen class models.

Learning Modality-Invariant Latent Representations for Generalized Zero-shot Learning

“…To this end, our motivations and formulations focus on the GZSL settings. Previous methods which focus on GZSL, e.g., CADA-VAE [37], PREN [43], COSOMO [4] and GDAN [19], did not report the performance on ZSL evaluations. Therefore, we do not conduct extensive evaluations on ZSL but only report the performance on the most challenging dataset used in this paper, i.e., SUN, to show that our model is also effective for ZSL.…”

“…To verify our proposed method, we compare it with both embedding methods: DeViSE [15], ESZSL [36], ALE [1], SAE [24], SJE [2], DEM [44]; and generative methods: f-CLSWGAN [41], GAZSL [45], cyc-CLSWGAN [14], SE [39], LisGAN [28], CADA-VAE [37], PREN [43] and COSMO [4]. The results of the compared methods are cited from the original papers 1 and the recent survey paper [42].…”

Generalised Zero-Shot Learning with Domain Classification in a Joint Semantic and Visual Space

“…These results are given for the data sets CUB, SUN, AWA1 and AWA2. We compare our approach with 12 leading GZSL methods, which are divided into three groups: semantic (SJE [24], ALE [25], LATEM [26], ES-ZSL [27], SYNC [12], DEVISE [2]), latent space learning (SAE [15], f-CLSWGAN [11], cycle-WGAN [3] and CADA-VAE [4]) and domain classification (CMT [6] and DAZSL [5]). The semantic group contains methods that only use the seen class visual and semantic samples to learn a transformation function from the visual to the semantic space, and classification is based on nearest neighbour classification in that semantic space.…”

“…Our second observation is that samples from unseen classes that are visually different from any of the seen classes, tend to be projected outside the distribution of seen classes [6]. Atzmon and Chechik [5] propose a general framework that combines domain expert classifiers, such as DAP [7] for unseen classes, and LAGO for the seen classes [5]. However, this method relies on the disjoint training of both experts models, and the assumption that unseen samples are projected outside the distribution of seen classes [6].…”

Dual Prototype Relaxation for Generalized Zero Shot Learning

“…However, direct search within all classes cannot well utilized the knowledge learned from the seen training samples, thus some Out-of-Domain (OoD) based methods are proposed to first classify the feature as seen or unseen, and then divide the GZSL problem into two sub-tasks: a conventional ZSL task and a fully supervised learning task. For example, some OoD [7,8] based methods define two classifiers to handle the seen and unseen domains separately. However, they all neglected that OoD detection is also a binary zeroshot classification, so it is unsuitable to use two totally different models for OoD detection and zero-shot classification respectively.…”