Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

Abstract

Zero‐shot learning (ZSL) addresses the novel object recognition problem by leveraging semantic embedding to transfer knowledge from seen categories to unseen categories. Generative ZSL models synthesize the visual features of unseen classes and convert ZSL task into a classical supervised learning problem. These generative ZSL models are trained by using the seen classes. Although promising progress has been achieved in the ZSL and generalized zero‐shot learning (GZSL) tasks. The existing approaches still suffer from a strong bias problem between unseen and seen classes, where unseen objects in the target domain tend to be recognized as seen classes in the source domain. To deal with the problem, we propose a novel named semantic consistent Wasserstein generative adversarial network (scWGAN), which uses a semantic reconstructor to reconstruct semantic embeddings from generated visual features by incorporating a novel Semantic Consistent Loss noted L rec. The Semantic Consistent Loss guides our proposed scWGAN to generate visual features that mirror the semantic relationships between seen and unseen classes. We also introduce a visual classifier to constrain visual feature generator. Extensive experiments show that the proposed approach is superior to previous state‐of‐the‐art works under both traditional ZSL and challenging GZSL settings on six popular data sets AWA1, AWA2, CUB, APY, and SUN.