Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification

Jayathilaka, Mirantha; Mu, Tingting; Sattler, Uli

doi:10.48550/arxiv.2109.09063

Cited by 1 publication

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• WordNet is the most widely used KG for augmenting both ZSL [2,5,36,62,63,88,98,103,105,183,188,192] and FSL [2,39,85,125,139,174]. As a large lexical database with several different relationships between words, such as synonym, hyponym, hypernym and meronym [122], it is often used to build task-specific class hierarchies, especially for image classification.…”

Section: Sub-kg Extractionmentioning

confidence: 99%

“…Input Mapping [85,114,125] Class Mapping [100] Joint Mapping [1,2,101,114,149,165,204,215,218,224] Data Augmentation…”

Section: Mapping-basedmentioning

confidence: 99%

“…In ZSL setting, the mapping, which is a linear transformation function, is learned by samples of the seen classes alone, while in the FSL setting, it is learned by samples of both seen and unseen classes. Jayathilaka et al [85] also learned a mapping function from the input features to the class embedding for few-shot image classification using labeled samples of both seen and unseen classes, where logical relationships such as class disjointness and class subsumption are represented by an ontology and considered in learning the class embeddings by an algorithm named EL Embedding [95]. Monka et al [125] investigated KG-augmented image classification under a transfer setting where a model trained by a large number of source domain samples is transferred to a target domain which has only a small number of labeled samples.…”

Section: Input Mappingmentioning

confidence: 99%

“…For few-shot image classification, the majority of the existing methods aim at utilizing the few-shot samples by e.g., meta learning, while the KG-aware studies often try to combine benefits from the KG external knowledge and the few-shot samples, which actually looks quite reasonable. Some of them simply extend their mapping-based models which are originally developed for zero-shot image classification by training with additional samples of the unseen classes (e.g., [2,85,101]), while some others further generate more data for unseen classes conditioned on KGs (e.g., [174]) or utilize KGs to transfer images features from seen classes to unseen classes (e.g., [39,139]). There are also some open benchmarks that can be used for evaluating KG-aware few-shot image classification.…”

Section: Computer Visionmentioning

confidence: 99%

See 3 more Smart Citations

Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Survey

Chen¹,

Geng²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning methods especially deep neural networks have achieved great success but many of them often rely on a number of labeled samples for training. In real-world applications, we often need to address sample shortage due to e.g., dynamic contexts with emerging prediction targets and costly sample annotation. Therefore, low-resource learning, which aims to learn robust prediction models with no enough resources (especially training samples), is now being widely investigated. Among all the low-resource learning studies, many prefer to utilize some auxiliary information in the form of Knowledge Graph (KG), which is becoming more and more popular for knowledge representation, to reduce the reliance on labeled samples. In this survey, we very comprehensively reviewed over 90 papers about KG-aware research for two major low-resource learning settings -zero-shot learning (ZSL) where new classes for prediction have never appeared in training, and few-shot learning (FSL) where new classes for prediction have only a small number of labeled samples that are available. We first introduced the KGs used in ZSL and FSL studies as well as the existing and potential KG construction solutions, and then systematically categorized and summarized KG-aware ZSL and FSL methods, dividing them into different paradigms such as the mapping-based, the data augmentation, the propagation-based and the optimization-based. We next presented different applications, including not only KG augmented prediction tasks in Computer Vision and Natural Language Processing (e.g., image classification, visual question answering, text classification and knowledge extraction), but also tasks for KG curation (e.g., inductive KG completion), and some typical evaluation resources for each task. We eventually discussed some challenges and future directions on aspects such as new learning and reasoning paradigms, and the construction of high quality KGs.

show abstract