Relation-Aware Compositional Zero-Shot Learning for Attribute-Object Pair Recognition

Xu, Ziwei; Wang, Guangzhi; Wong, Yongkang; Kankanhalli, Mohan

doi:10.1109/tmm.2021.3104411

Cited by 14 publications

(5 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Karthik et al [21] further improve simple primitives by estimating the feasibility of each composition through external knowledge to eliminate impossible state-object pairs. Different from [21], [40], our work estimates feasibility with attention mechanism and implies feasibility in the form of probability to enhance simple primitive learning.…”

Section: Open-world Compositional Zero-shot Learningmentioning

confidence: 99%

“…For example, Li et al [19] mask impossible compositions by computing pair probability based on the distance between state and object categories. Xu et al [40] adopt a keyquery-based attention mechanism to capture the correlation between primitive concepts in a graph to pass messages selectively. In OW-CZSL, similar to [19], Mancini et al [22], [29] propose to utilize the graph structure to model the dependence between state, object, and compositions.…”

Section: Open-world Compositional Zero-shot Learningmentioning

confidence: 99%

See 1 more Smart Citation

Simple Primitives with Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-shot Learning

Liu¹,

Li²,

Yao³

et al. 2022

Preprint

View full text Add to dashboard Cite

The task of Compositional Zero-Shot Learning (CZSL) is to recognize images of novel state-object compositions that are absent during the training stage. Previous methods of learning compositional embedding have shown effectiveness in closed-world CZSL. However, in Open-World CZSL (OW-CZSL), their performance tends to degrade significantly due to the large cardinality of possible compositions. Some recent works separately predict simple primitives (i.e., states and objects) to reduce cardinality. However, they consider simple primitives as independent probability distributions, ignoring the heavy dependence between states, objects, and compositions. In this paper, we model the dependence of compositions via feasibility and contextuality. Feasibility-dependence refers to the unequal feasibility relations between simple primitives, e.g., hairy is more feasible with cat than with building in the real world. Contextuality-dependence represents the contextual variance in images, e.g., cat shows diverse appearances under the state of dry and wet. We design Semantic Attention (SA) and generative Knowledge Disentanglement (KD) to learn the dependence of feasibility and contextuality, respectively. SA captures semantics in compositions to alleviate impossible predictions, driven by the visual similarity between simple primitives. KD disentangles images into unbiased feature representations, easing contextual bias in predictions. Moreover, we complement the current compositional probability model with feasibility and contextuality in a compatible format. Finally, we conduct comprehensive experiments to analyze and validate the superior or competitive performance of our model, Semantic Attention and knowledge Disentanglement guided Simple Primitives (SAD-SP), on three widely-used benchmark OW-CZSL datasets.

show abstract

Section: Open-world Compositional Zero-shot Learningmentioning

confidence: 99%

Section: Open-world Compositional Zero-shot Learningmentioning

confidence: 99%

Simple Primitives with Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-shot Learning

Liu¹,

Li²,

Yao³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Li et al [15] disentangle attributes and objects with reversed attention. The second strategy directly predicts compositions by aligning images and textual labels in a shared space and searching for most similar compositions [13,26,35,42]. For example, Nagarajan et al [26] build a composition space by simulating all the visual changes of attributes performed on objects.…”

Section: Related Workmentioning

confidence: 99%

Diversity-Boosted Generalization-Specialization Balancing for Zero-Shot Learning

Liu

Chang

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

The objective of Active Learning is to strategically label a subset of the dataset to maximize performance within a predetermined labeling budget. In this study, we harness features acquired through self-supervised learning. We introduce a straightforward yet potent metric, Cluster Distance Difference, to identify diverse data. Subsequently, we introduce a novel framework, Balancing Active Learning (BAL), which constructs adaptive sub-pools to balance diverse and uncertain data. Our approach outperforms all established active learning methods on widely recognized benchmarks by 1.20%. Moreover, we assess the efficacy of our proposed framework under extended settings, encompassing both larger and smaller labeling budgets. Experimental results demonstrate that, when labeling 80% of the samples, the performance of the current SOTA method declines by 0.74%, whereas our proposed BAL achieves performance comparable to the full dataset. Codes are available at https://github.com/JulietLJY/BAL.

show abstract

“…The major challenge behind the CZSL lies in how to model the interactions between state and object primitives and extrapolate seen compositions to unseen ones. Existing methods mainly focus on learning a shared embedding space for object-state compositions (Li et al 2020;Naeem et al 2021;Nagarajan and Grauman 2018;Khan et al 2023) or compositional attribute and object classifiers (Purushwalkam et al 2019;Misra, Gupta, and Hebert 2017;Xu et al 2022;Yang et al 2022).…”

Section: Introductionmentioning

confidence: 99%

ProCC: Progressive Cross-Primitive Compatibility for Open-World Compositional Zero-Shot Learning

Huo,

Xu,

Guo

et al. 2024

AAAI

View full text Add to dashboard Cite

Open-World Compositional Zero-shot Learning (OW-CZSL) aims to recognize novel compositions of state and object primitives in images with no priors on the compositional space, which induces a tremendously large output space containing all possible state-object compositions. Existing works either learn the joint compositional state-object embedding or predict simple primitives with separate classifiers. However, the former method heavily relies on external word embedding methods, and the latter ignores the interactions of interdependent primitives, respectively. In this paper, we revisit the primitive prediction approach and propose a novel method, termed Progressive Cross-primitive Compatibility (ProCC), to mimic the human learning process for OW-CZSL tasks. Specifically, the cross-primitive compatibility module explicitly learns to model the interactions of state and object features with the trainable memory units, which efficiently acquires cross-primitive visual attention to reason high-feasibility compositions, without the aid of external knowledge. Moreover, to alleviate the invalid cross-primitive interactions, especially for partial-supervision conditions (pCZSL), we design a progressive training paradigm to optimize the primitive classifiers conditioned on pre-trained features in an easy-to-hard manner. Extensive experiments on three widely used benchmark datasets demonstrate that our method outperforms other representative methods on both OW-CZSL and pCZSL settings by large margins.

show abstract

Relation-Aware Compositional Zero-Shot Learning for Attribute-Object Pair Recognition

Cited by 14 publications

References 74 publications

Simple Primitives with Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-shot Learning

Simple Primitives with Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-shot Learning

Diversity-Boosted Generalization-Specialization Balancing for Zero-Shot Learning

ProCC: Progressive Cross-Primitive Compatibility for Open-World Compositional Zero-Shot Learning

Contact Info

Product

Resources

About