A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts

Zhu, Yizhe; Elhoseiny, Mohamed; Liu, Bingchen; Peng, Xi; Elgammal, Ahmed

doi:10.1109/cvpr.2018.00111

Cited by 385 publications

(432 citation statements)

References 31 publications

Supporting

Mentioning

429

Contrasting

Order By: Relevance

“…However, the accuracy of unseen samples would dramatically drop in GZSL. As we mentioned before, the unseen samples accuracy of GAZSL [37] drops from 68.2% (ZSL) to 19.2% (GZSL). To the best of our knowledge, there is no previous work explicitly addressing the feature confusion issue in generative ZSL.…”

Section: Generative Zero-shot Learningmentioning

confidence: 65%

“…Thus, ZSL tasks involve a shared semantic space and a visual space where seen samples and unseen ones have distinctive data distributions. According to the working mechanisms, existing ZSL methods can be grouped into either embedding methods [2,6,33,36] or generative methods [17,29,31,37]. Specifically, embedding methods learn a visual-to-semantic embedding space, or a semantic-to-visual embedding space, or a shared intermediate embedding space where the two domains are connected.…”

Section: Related Work 21 Zero-shot Learningmentioning

confidence: 99%

“…For instance, Xian et al [31] propose a feature generating network for ZSL by deploying conditional WGAN. Zhu et al [37] introduce a feature synthesizing network by GANs constrained by a visual pivot. Verma et al [29] propose to handle GZSL by synthesized samples.…”

Section: Generative Zero-shot Learningmentioning

confidence: 99%

“…This phenomenon can also be reflected by quantitative results. Taking another state-of-the-art GAZSL [37] as an example, the accuracy of unseen samples on CUB is 55.8% when only unseen instances are tested. However, this accuracy drops to 23.9% when seen categories are also involved in the testing set.…”

Section: The Boundary Lossmentioning

confidence: 99%

“…Embedding methods [7,34,36] generally learn a visual-to-semantic, semanticto-visual or joint embedding space so that both the seen and unseen samples can be compared in the shared space. Generative methods [29,31,37] aim to synthesize unseen features directly from the semantic descriptions. Recently, generative methods have achieved state-of-the-art results by converting ZSL into a general supervised learning problem.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Alleviating Feature Confusion for Generative Zero-shot Learning

Jing

Lü

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Lately, generative adversarial networks (GANs) have been successfully applied to zero-shot learning (ZSL) and achieved state-of-theart performance. By synthesizing virtual unseen visual features, GAN-based methods convert the challenging ZSL task into a supervised learning problem. However, GAN-based ZSL methods have to train the generator on the seen categories and further apply it to unseen instances. An inevitable issue of such a paradigm is that the synthesized unseen features are prone to seen references and incapable to reflect the novelty and diversity of real unseen instances. In a nutshell, the synthesized features are confusing. One cannot tell unseen categories from seen ones using the synthesized features. As a result, the synthesized features are too subtle to be classified in generalized zero-shot learning (GZSL) which involves both seen and unseen categories at the test stage. In this paper, we first introduce the feature confusion issue. Then, we propose a new feature generating network, named alleviating feature confusion GAN (AFC-GAN), to challenge the issue. Specifically, we present a boundary loss which maximizes the decision boundary of seen categories and unseen ones. Furthermore, a novel metric named feature confusion score (FCS) is proposed to quantify the feature confusion. Extensive experiments on five widely used datasets verify that our method is able to outperform previous state-of-the-arts under both ZSL and GZSL protocols.

show abstract

Section: Generative Zero-shot Learningmentioning

confidence: 65%

Section: Related Work 21 Zero-shot Learningmentioning

confidence: 99%

Section: Generative Zero-shot Learningmentioning

confidence: 99%

Section: The Boundary Lossmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Alleviating Feature Confusion for Generative Zero-shot Learning

Jing

Lü

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

Knowledge Distillation‐Based Zero‐Shot Learning for Process Fault Diagnosis

Liu,

Huang,

Jia

2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Data‐driven deep learning is effective in diagnosing known faults, but not so well when new or unknown faults occur. With unknown faults as a zero‐shot learning problem, this article proposes a method for detecting and isolating unknown faults based on knowledge distillation within a teacher–student framework. Process data and image data are equivalent in their spatiotemporal dimensions, and convolutional neural networks are selected as the teacher model, pretrained on image data. Information under both normal and fault conditions is then effectively extracted from process data by the well‐trained teacher model. Subsequently, knowledge distillation is used to transfer only the data of normal conditions in the teacher model to the student model. When an unknown fault arises, there exist differences between the information extracted by the teacher model and the student model. Contributions of variables to faults are calculated by quantifying these differences through gradients, thereby isolating the unknown fault. Finally, compared with a series of baseline methods and two state‐of‐the‐art methods, the proposed method improves fault diagnosis accuracy by 3.08% to 26.13% in the Tennessee Eastman process and by 3.48% to 41.45% in the sour water treatment process. Additionally, the physical consistency of fault isolation is visually assessed.

show abstract

Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

Zhao

Liu

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

Zero‐shot learning (ZSL) addresses the novel object recognition problem by leveraging semantic embedding to transfer knowledge from seen categories to unseen categories. Generative ZSL models synthesize the visual features of unseen classes and convert ZSL task into a classical supervised learning problem. These generative ZSL models are trained by using the seen classes. Although promising progress has been achieved in the ZSL and generalized zero‐shot learning (GZSL) tasks. The existing approaches still suffer from a strong bias problem between unseen and seen classes, where unseen objects in the target domain tend to be recognized as seen classes in the source domain. To deal with the problem, we propose a novel named semantic consistent Wasserstein generative adversarial network (scWGAN), which uses a semantic reconstructor to reconstruct semantic embeddings from generated visual features by incorporating a novel Semantic Consistent Loss noted L rec. The Semantic Consistent Loss guides our proposed scWGAN to generate visual features that mirror the semantic relationships between seen and unseen classes. We also introduce a visual classifier to constrain visual feature generator. Extensive experiments show that the proposed approach is superior to previous state‐of‐the‐art works under both traditional ZSL and challenging GZSL settings on six popular data sets AWA1, AWA2, CUB, APY, and SUN.

show abstract

A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts

Cited by 385 publications

References 31 publications

Alleviating Feature Confusion for Generative Zero-shot Learning

Alleviating Feature Confusion for Generative Zero-shot Learning

Knowledge Distillation‐Based Zero‐Shot Learning for Process Fault Diagnosis

Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

Contact Info

Product

Resources

About