Translational Concept Embedding for Generalized Compositional Zero-shot Learning

Abstract

Generalized compositional zero-shot learning means to learn composed concepts of attribute-object pairs in a zero-shot fashion, where a model is trained on a set of seen concepts and tested on a combined set of seen and unseen concepts. This task is very challenging because of not only the gap between seen and unseen concepts but also the contextual dependency between attributes and objects. This paper introduces a new approach, termed translational concept embedding, to solve these two difficulties in a unified framework. It models the effect of applying an attribute to an object as adding a translational attribute feature to an object prototype. We explicitly take into account of the contextual dependency between attributes and objects by generating translational attribute features conditionally dependent on the object prototypes. Furthermore, we design a ratio variance constraint loss to promote the model's generalization ability on unseen concepts. It regularizes the distances between concepts by utilizing knowledge from their pretrained word embeddings. We evaluate the performance of our model under both the unbiased and biased concept classification tasks, and show that our model is able to achieve good balance in predicting unseen and seen concepts.

Leveraging Sub-class Discimination for Compositional Zero-Shot Learning

“…Differently, if a person has seen the images of a 'colorful car' and an 'old building', he can understand the concept of a 'colorful building' even without having seen it previously. Therefore, the researchers are committed to integrating such Compositional Zero-shot Learning (CZSL) capability to the computer vision systems (Nan et al 2019;Huang et al 2021).…”