Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learning

Vyas, Maunil R.; Venkateswara, Hemanth; Panchanathan, Sethuraman

doi:10.1007/978-3-030-58577-8_5

Cited by 107 publications

(46 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed two HarS based inductive methods, HarS-WGAN and HarS-VAEGAN are compared with 18 stateof-the-art inductive methods: DEVISE [4], LATEM [17], SAE [49], DEM [22], LiGAN [28], ABP [30], TCN [24], OCD-GZSL [50], APNet [51], DE-VAE [52], LsrGAN [53], f-CSLWGAN [11], DASCN [27], DVBE [9], DAZLE [54], TF-VAEGAN [13], CF-GZSL [19], GCM-CF [37]. The proposed two HarST based transductive methods, HarST-DEM and HarST-WGAN are compared with 14 state-ofthe-art transductive methods: ALE-tran [16], GFZSL [55], DSRL [56], QFSL [40], GMN [57], f-VAEGAN-D2 [44], GXE [42], SABR-T [29], PREN [41], VSC [58], DTN [59], ADA [60], SDGN [45], TF-VAEGAN [13].…”

Section: Datasets and Comparative Methodsmentioning

confidence: 99%

HardBoost: Boosting Zero-Shot Learning with Hard Classes

Liu¹,

Hu²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

This work is a systematical analysis on the so-called hard class problem in zero-shot learning (ZSL), that is, some unseen classes disproportionally affect the ZSL performances than others, as well as how to remedy the problem by detecting and exploiting hard classes. At first, we report our empirical finding that the hard class problem is a ubiquitous phenomenon and persists regardless of used specific methods in ZSL. Then, we find that high semantic affinity among unseen classes is a plausible underlying cause of hardness and design two metrics to detect hard classes. Finally, two frameworks are proposed to remedy the problem by detecting and exploiting hard classes, one under inductive setting, the other under transductive setting. The proposed frameworks could accommodate most existing ZSL methods to further significantly boost their performances with little efforts. Extensive experiments on three popular benchmarks demonstrate the benefits by identifying and exploiting the hard classes in ZSL.

show abstract

Section: Datasets and Comparative Methodsmentioning

confidence: 99%

HardBoost: Boosting Zero-Shot Learning with Hard Classes

Liu¹,

Hu²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…There are some knowledge graph embedding algorithms based on GAN and Zero-Shot Learning [157]. Vyas et al [158] proposed a Generalized Zero-Shot learning model, which can find unseen semantic in knowledge graphs. 5) Graph Spatial-Temporal Networks: Graph spatialtemporal networks simultaneously capture the spatial and temporal dependence of graphs.…”

Section: ) Graph Generative Networkmentioning

confidence: 99%

Graph Learning: A Survey

Xia

Sun

et al. 2021

IEEE Trans. Artif. Intell.

297

View full text Add to dashboard Cite

Graphs are widely used as a popular representation of the network structure of connected data. Graph data can be found in a broad spectrum of application domains such as social systems, ecosystems, biological networks, knowledge graphs, and information systems. With the continuous penetration of artificial intelligence technologies, graph learning (i.e., machine learning on graphs) is gaining attention from both researchers and practitioners. Graph learning proves effective for many tasks, such as classification, link prediction, and matching. Generally, graph learning methods extract relevant features of graphs by taking advantage of machine learning algorithms. In this survey, we present a comprehensive overview on the state-of-the-art of graph learning. Special attention is paid to four categories of existing graph learning methods, including graph signal processing, matrix factorization, random walk, and deep learning. Major models and algorithms under these categories are reviewed respectively. We examine graph learning applications in areas such as text, images, science, knowledge graphs, and combinatorial optimization. In addition, we discuss several promising research directions in this field.

show abstract

“…In addition, they deploy the average representation of all samples from an unseen class defining the soul sample of the class to reduce the noise in the predictions. Vyas et al [141] propose LsrGAN, a generative model that leverages the semantic relationship between seen and unseen categories and explicitly performs knowledge transfer by incorporating a novel semantic regularized loss (SR-Loss). Knowing the inter-class relationships in the semantic space helps to impose the same relationship constraints among the generated visual features.…”

Section: Semantic-visual Features Extractorsmentioning

confidence: 99%

A survey on visual transfer learning using knowledge graphs

Monka

Halilaj

Rettinger

2022

View full text Add to dashboard Cite

The information perceived via visual observations of real-world phenomena is unstructured and complex. Computer vision (CV) is the field of research that attempts to make use of that information. Recent approaches of CV utilize deep learning (DL) methods as they perform quite well if training and testing domains follow the same underlying data distribution. However, it has been shown that minor variations in the images that occur when these methods are used in the real world can lead to unpredictable and catastrophic errors. Transfer learning is the area of machine learning that tries to prevent these errors. Especially, approaches that augment image data using auxiliary knowledge encoded in language embeddings or knowledge graphs (KGs) have achieved promising results in recent years. This survey focuses on visual transfer learning approaches using KGs, as we believe that KGs are well suited to store and represent any kind of auxiliary knowledge. KGs can represent auxiliary knowledge either in an underlying graph-structured schema or in a vector-based knowledge graph embedding. Intending to enable the reader to solve visual transfer learning problems with the help of specific KG-DL configurations we start with a description of relevant modeling structures of a KG of various expressions, such as directed labeled graphs, hypergraphs, and hyper-relational graphs. We explain the notion of feature extractor, while specifically referring to visual and semantic features. We provide a broad overview of knowledge graph embedding methods and describe several joint training objectives suitable to combine them with high dimensional visual embeddings. The main section introduces four different categories on how a KG can be combined with a DL pipeline: 1) Knowledge Graph as a Reviewer; 2) Knowledge Graph as a Trainee; 3) Knowledge Graph as a Trainer; and 4) Knowledge Graph as a Peer. To help researchers find meaningful evaluation benchmarks, we provide an overview of generic KGs and a set of image processing datasets and benchmarks that include various types of auxiliary knowledge. Last, we summarize related surveys and give an outlook about challenges and open issues for future research.

show abstract

Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learning

Cited by 107 publications

References 35 publications

HardBoost: Boosting Zero-Shot Learning with Hard Classes

HardBoost: Boosting Zero-Shot Learning with Hard Classes

Graph Learning: A Survey

A survey on visual transfer learning using knowledge graphs

Contact Info

Product

Resources

About