Towards Unbiased Multi-Label Zero-Shot Learning With Pyramid and Semantic Attention

Abstract

This paper investigates a challenging problem of zero-shot learning in the multi-label scenario (MLZSL), wherein, the model is trained to recognize multiple unseen classes within a sample (e.g., an image) based on seen classes and auxiliary knowledge, e.g., semantic information. Existing methods usually resort to analyzing the relationship of various seen classes residing in a sample from the dimension of spatial or semantic characteristics, and transfer the learned model to unseen ones. But they ignore the effective integration of local and global features. That is, in the process of inferring unseen classes, global features represent the principal direction of the image in the feature space, while local features should maintain uniqueness within a certain range. This integrated neglect will make the model lose its grasp of the main components of the image. Relying only on the local existence of seen classes during the inference stage introduces unavoidable bias. In this paper, we propose a novel and effective group bienhancement framework for MLZSL, dubbed GBE-MLZSL, to fully make use of such properties and enable a more accurate and robust visual-semantic projection. Specifically, we split the feature maps into several feature groups, of which each feature group can be trained independently with the Local Information Distinguishing Module (LID) to ensure uniqueness. Meanwhile, a Global Enhancement Module (GEM) is designed to preserve the principal direction. Besides, a static graph structure is designed to construct the correlation of local features. Experiments on largescale MLZSL benchmark datasets NUS-WIDE and Open-Images-v4 demonstrate that the proposed GBE-MLZSL outperforms other state-of-the-art methods with large margins.

Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning

“…Compositional Zero-Shot Learning. CZSL [9,20,25,26,28] is a task similar to how humans can imagine and discriminate unseen concepts according to the concepts they have learned, which is a significant branch of ZSL [5,10,11,15,17,21,22,40].…”

Graph Knows Unknowns: Reformulate Zero-Shot Learning as Sample-Level Graph Recognition

“…Recent years have seen a rise of interest in zero-shot learning (ZSL) which imitates the human ability to recognize unseen classes without observing real samples (Kodirov, Xiang, and Gong 2017;Yu et al 2018;Zhu et al 2019;Chen et al 2021c;Liu et al 2022;Kim, Shim, and Shim 2022;Su et al 2022;Khan et al 2023). Specifically, ZSL takes utilization of seen classes with labeled samples and auxiliary knowledge between seen and unseen classes to achieve recognition.…”