Unseen Classes at a Later Time? No Problem

Abstract

Recent progress towards learning from limited supervision has encouraged efforts towards designing models that can recognize novel classes at test time (generalized zeroshot learning or GZSL). GZSL approaches assume knowledge of all classes, with or without labeled data, beforehand. However, practical scenarios demand models that are adaptable and can handle dynamic addition of new seen and unseen classes on the fly (i.e continual generalized zero-shot learning or CGZSL). One solution is to sequentially retrain and reuse conventional GZSL methods, however, such an approach suffers from catastrophic forgetting leading to suboptimal generalization performance. A few recent efforts towards tackling CGZSL have been limited by difference in settings, practicality, data splits and protocols followed -inhibiting fair comparison and a clear direction forward. Motivated from these observations, in this work, we firstly consolidate the different CGZSL setting variants and propose a new Online-CGZSL setting which is more practical and flexible. Secondly, we introduce a unified feature-generative framework for CGZSL that leverages bi-directional incremental alignment to dynamically adapt to addition of new classes, with or without labeled data, that arrive over time in any of these CGZSL settings.Our comprehensive experiments and analysis on five benchmark datasets and comparison with baselines show that our approach consistently outperforms existing methods, especially on the more practical Online setting.

Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion

“…Zero-shot Learning The cross-modality data translation problem without access to the source modal data leads to a Zero-shot-Learning-based crossmodality data translation problem. One challenge of learning-based methods is that their modeling ability is restricted when dealing with unseen data classes [66,9,34]. Zero-shot-Learning is a robust learning scheme to deal with such cases when training and test classes are disjoint.…”

Meta-Learning for Generalized Zero-Shot Learning

“…As such, there are clear connections between the two problem settings. Some recent works [12,13,20,29,38,59] have drawn increasing attention towards continual zero-shot learning (CZSL). For example, [59] considers a task incremental learning setting, where task ID for each sample is provided during train and test, leading to an easier and perhaps less realistic setting than class incremental learning.…”

“…This setting is referred to as Fixed Continual GZSL. Meanwhile, [12,13,29] proposed a replay-based approach, showing state-of-the-art results in a more realistic setting for CSZL called Dynamic Continual GZSL, where the attributes for unseen classes of future tasks are not known a priori.…”

“…For the non-continual (standard) ZSL and GZSL baselines, please refer directly to tables 3 and 4 for a variety of non-continual baselines we considered. To the best of our knowledge, continual ZSL has mainly been explored in [3,12,20,29,59]. For comparison, we consider CZSL-CV+res and CZSL-CA+res [12], which use conditional variational auto-encoders (VAE) and CADA [48] with a memory reservoir, respectively.…”

“…NM-ZSL [20] is also a competitive continual ZSL baseline and other continual ZSL baselines include Seq-CVAE [40] and Seq-CADA [48], which sequentially train a VAE over tasks to augment the training dataset using replay. The recent work UCLT [29] uses a generative model and reply to solve the CZSL problem.…”