FREE: Feature Refinement for Generalized Zero-Shot Learning

Chen, Shiming; Wang, Wenjie; Xia, Beihao; Peng, Qinmu; You, Xinge; Zheng, Feng; Shao, Ling

doi:10.1109/iccv48922.2021.00019

Cited by 148 publications

(59 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They usually extract global visual features from pre-trained or end-to-end trainable networks, e.g., ResetNet [35]. Note that end-to-end models achieve better performance than pre-trained ones because they fine-tune the visual features, thus the cross-dataset bias between ImageNet and ZSL benchmarks is alleviated [8], [26].…”

Section: Zero-shot Learningmentioning

confidence: 99%

“…Since the cross-dataset bias between ImageNet and ZSL benchmarks potentially limits the quality of visual feature extraction [26], [38], we first propose a feature augmentation encoder to refine the visual features of ZSL benchmarks. In addition, previous ZSL methods simply flatten the grid features U (x) ∈ R H×W ×C (extracted by a CNN backbone) of a single image into a feature vector, which is further applied to generative models or embedding learning.…”

Section: Feature Augmentation Encodermentioning

confidence: 99%

“…1. https://pytorch.org/ E-PGN [66], SDGZSL [38], FREE [26], HSVA [25]) and nongenerative methods (e.g., SP-AEN [69], PQZSL [70], IIR [71], TCN [72], DVBE [73], DAZLE [29]), to demonstrate its effectiveness and advantages.…”

Section: Comparison With State-of-the-artmentioning

confidence: 99%

“…However, these embedding-based methods inevitably overfit to seen classes under the GZSL setting (known as the bias problem), since the embedding is only learned by seen class samples. To mitigate this bias problem, many generative ZSL methods have been proposed to synthesize feature samples for unseen classes by leveraging generative models (e.g., variational autoencoders (VAEs) [23], [24], [25], generative adversarial nets (GANs) [8], [16], [26], and generative flows [27]) for data augmentation. Thus the generative ZSL methods can compensate for the lack of training samples of unseen classes and convert the ZSL task into a supervised classification task.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, most of aforementioned methods directly use feature extraction models (e.g., ResNet [35]) pre-trained on ImageNet [36] alone, ignoring the cross-dataset bias between ImageNet and GZSL benchmarks (e.g., CUB [37]). Such a bias inevitably results in poor-quality visual features in which not all the dimensions are semantically related to the pre-defined attributes for ZSL tasks, which potentially limits the recognition performance on both seen and unseen classes [26], [38]. Thus, properly improving the transferability and localizing the object attributes of visual features for enabling significant visual-semantic interaction in ZSL has become very necessary.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

Chen¹,

Hong²,

Xie³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Zero-shot learning (ZSL) tackles the novel class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Semantic knowledge is typically represented by attribute descriptions shared between different classes, which act as strong priors for localizing object attributes that represent discriminative region features, enabling significant and sufficient visual-semantic interaction for advancing ZSL. Existing attention-based models have struggled to learn inferior region features in a single image by solely using unidirectional attention, which ignore the transferability and discriminative attribute localization of visual features. In this paper, we propose a cross attribute-guided Transformer network, termed TransZero++, to refine visual features and learn accurate attribute localization for semantic-augmented visual embedding representations in ZSL. TransZero++ consists of an attribute→visual Transformer sub-net (AVT) and a visual→attribute Transformer sub-net (VAT). Specifically, AVT first takes a feature augmentation encoder to alleviate the cross-dataset bias between ImageNet and ZSL benchmarks, and improves the transferability of visual features by reducing the entangled relative geometry relationships among region features. Then, an attribute→visual decoder is employed to localize the image regions most relevant to each attribute in a given image for attribute-based visual feature representations. Analogously, VAT uses the similar feature augmentation encoder to refine the visual features, which are further applied in visual→attribute decoder to learn visual-based attribute features. By further introducing feature-level and prediction-level semantical collaborative losses, the two attribute-guided transformers teach each other to learn semantic-augmented visual embeddings via semantical collaborative learning. Finally, the semantic-augmented visual embeddings learned by AVT and VAT are fused to conduct desirable visual-semantic interaction cooperated with semantic vectors for ZSL classification. Extensive experiments show that TransZero++ achieves the new state-of-the-art results on three golden and challenging ZSL benchmarks. The codes are available at: https://github.com/shiming-chen/TransZero_pp.

show abstract

Section: Zero-shot Learningmentioning

confidence: 99%

Section: Feature Augmentation Encodermentioning

confidence: 99%

Section: Comparison With State-of-the-artmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

Chen¹,

Hong²,

Xie³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Knowledge Distillation‐Based Zero‐Shot Learning for Process Fault Diagnosis

Liu,

Huang,

Jia

2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Data‐driven deep learning is effective in diagnosing known faults, but not so well when new or unknown faults occur. With unknown faults as a zero‐shot learning problem, this article proposes a method for detecting and isolating unknown faults based on knowledge distillation within a teacher–student framework. Process data and image data are equivalent in their spatiotemporal dimensions, and convolutional neural networks are selected as the teacher model, pretrained on image data. Information under both normal and fault conditions is then effectively extracted from process data by the well‐trained teacher model. Subsequently, knowledge distillation is used to transfer only the data of normal conditions in the teacher model to the student model. When an unknown fault arises, there exist differences between the information extracted by the teacher model and the student model. Contributions of variables to faults are calculated by quantifying these differences through gradients, thereby isolating the unknown fault. Finally, compared with a series of baseline methods and two state‐of‐the‐art methods, the proposed method improves fault diagnosis accuracy by 3.08% to 26.13% in the Tennessee Eastman process and by 3.48% to 41.45% in the sour water treatment process. Additionally, the physical consistency of fault isolation is visually assessed.

show abstract

A review on multimodal zero‐shot learning

Cao

Sun

et al. 2023

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Multimodal learning provides a path to fully utilize all types of information related to the modeling target to provide the model with a global vision. Zero-shot learning (ZSL) is a general solution for incorporating prior knowledge into data-driven models and achieving accurate class identification. The combination of the two, known as multimodal ZSL (MZSL), can fully exploit the advantages of both technologies and is expected to produce models with greater generalization ability. However, the MZSL algorithms and applicationshave not yet been thoroughly investigated and summarized. This study fills this gap by providing an objective overview of MZSL's definition, typical algorithms, representative applications, and critical issues. This article will not only provide researchers in this field with a comprehensive perspective, but it will also highlight several promising research directions.

show abstract

FREE: Feature Refinement for Generalized Zero-Shot Learning

Cited by 148 publications

References 42 publications

TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

Knowledge Distillation‐Based Zero‐Shot Learning for Process Fault Diagnosis

A review on multimodal zero‐shot learning

Contact Info

Product

Resources

About