Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases

Liu, Yu; Yao, Quanming; Li, Yong

doi:10.1145/3366423.3380188

Cited by 59 publications

(66 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To preserve the whole expressivity of the KG, a set of new KGE-Methods are developed to directly operate on hypergraphs and hyper-relational graphs. Some of the methods that deal with hypergraphs are HEBE [52], HGE [157], Hyper2vec [59], HNN [36], HCN [154], DHNE [134], HHNE [9], Hyper-SAGNN [164], HypE [35] and methods that embedd hyper-relational graphs are for instance m-TransH [151], HSimple [35], RAE [163], GETD [84], TuckER [7], NaLP [51], HINGE [116], StarE [40].…”

Section: Kge-methods -Input Typementioning

confidence: 99%

A survey on visual transfer learning using knowledge graphs

Monka

Halilaj

Rettinger

2022

View full text Add to dashboard Cite

The information perceived via visual observations of real-world phenomena is unstructured and complex. Computer vision (CV) is the field of research that attempts to make use of that information. Recent approaches of CV utilize deep learning (DL) methods as they perform quite well if training and testing domains follow the same underlying data distribution. However, it has been shown that minor variations in the images that occur when these methods are used in the real world can lead to unpredictable and catastrophic errors. Transfer learning is the area of machine learning that tries to prevent these errors. Especially, approaches that augment image data using auxiliary knowledge encoded in language embeddings or knowledge graphs (KGs) have achieved promising results in recent years. This survey focuses on visual transfer learning approaches using KGs, as we believe that KGs are well suited to store and represent any kind of auxiliary knowledge. KGs can represent auxiliary knowledge either in an underlying graph-structured schema or in a vector-based knowledge graph embedding. Intending to enable the reader to solve visual transfer learning problems with the help of specific KG-DL configurations we start with a description of relevant modeling structures of a KG of various expressions, such as directed labeled graphs, hypergraphs, and hyper-relational graphs. We explain the notion of feature extractor, while specifically referring to visual and semantic features. We provide a broad overview of knowledge graph embedding methods and describe several joint training objectives suitable to combine them with high dimensional visual embeddings. The main section introduces four different categories on how a KG can be combined with a DL pipeline: 1) Knowledge Graph as a Reviewer; 2) Knowledge Graph as a Trainee; 3) Knowledge Graph as a Trainer; and 4) Knowledge Graph as a Peer. To help researchers find meaningful evaluation benchmarks, we provide an overview of generic KGs and a set of image processing datasets and benchmarks that include various types of auxiliary knowledge. Last, we summarize related surveys and give an outlook about challenges and open issues for future research.

show abstract

Section: Kge-methods -Input Typementioning

confidence: 99%

A survey on visual transfer learning using knowledge graphs

Monka

Halilaj

Rettinger

2022

View full text Add to dashboard Cite

show abstract

“…Recently, GETD (Liu et al, 2020) extended TuckER (Balazevic et al, 2019) tensor factorization approach for n-ary relational facts. However, the model still expects only one relation in a fact and is not able to process facts of different arity in one dataset, e.g., 3-ary and 4-ary facts have to be split and trained separately.…”

Section: Related Workmentioning

confidence: 99%

“…Recent approaches (Guan et al, 2019;Liu et al, 2020;Rosso et al, 2020) for embedding hyperrelational KGs often use WikiPeople and JF17K as benchmarking datasets. We advocate that those datasets can not fully capture the task complexity.…”

Section: Wd50k Datasetmentioning

confidence: 99%

“…Existing representation learning approaches for such graphs largely treat a hyper-relational fact as an n-ary (n>2) composed relation (e.g., educatedAt academicDegree) (Zhang et al, 2018;Liu et al, 2020) losing entity-relation attribution; ignoring the semantic difference between a triple relation (educatedAt) and qualifier relation (academicDegree) (Guan et al, 2019), or decomposing a hyper-relational instance into multiple quintuples comprised of a triple and one qualifier key-value pair (Rosso et al, 2020). In this work, we propose an alternate graph representation learning mechanism capable of encoding hyper-relational KGs with arbitrary number of qualifiers, while keeping the semantic roles of qualifiers and triples intact.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Message Passing for Hyper-Relational Knowledge Graphs

Galkin¹,

Trivedi²,

Maheshwari³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Hyper-relational knowledge graphs (KGs) (e.g., Wikidata) enable associating additional key-value pairs along with the main triple to disambiguate, or restrict the validity of a fact. In this work, we propose a message passing based graph encoder -STARE capable of modeling such hyper-relational KGs. Unlike existing approaches, STARE can encode an arbitrary number of additional information (qualifiers) along with the main triple while keeping the semantic roles of qualifiers and triples intact. We also demonstrate that existing benchmarks for evaluating link prediction (LP) performance on hyper-relational KGs suffer from fundamental flaws and thus develop a new Wikidata-based dataset -WD50K. Our experiments demonstrate that STARE based LP model outperforms existing approaches across multiple benchmarks. We also confirm that leveraging qualifiers is vital for link prediction with gains up to 25 MRR points compared to triple-based representations.

show abstract

“…Each hyper-relational fact has a base triple (h, r, t) and additional key-value (relationentity) pairs (k, v) (e.g., {(academic major, physical), (academic degree, Master of Science)}). A line of work converts hyper-relational facts to n-ary meta-relations r(e 1 , ..., e n ) and leverages translational distance embedding [66,70], spatiotranslational embedding [1], tensor factorization [44] for modeling. Other approaches directly learn hyper-relational facts in their original form using various techniques, including convolutional neural networks, graph neural networks, and transformer models [53,24].…”

Section: B Modeling N-ary Factsmentioning

confidence: 99%

Learning Instance-Level N-Ary Semantic Knowledge At Scale For Robots Operating in Everyday Environments

Liu¹,

Bansal²,

Daruna³

et al. 2021

Robotics: Science and Systems XVII

View full text Add to dashboard Cite

Robots operating in everyday environments need to effectively perceive, model, and infer semantic properties of objects. Existing knowledge reasoning frameworks only model binary relations between an object's class label and its semantic properties, unable to collectively reason about object properties detected by different perception algorithms and grounded in diverse sensory modalities. We bridge the gap between multimodal perception and knowledge reasoning by introducing an n-ary representation that models complex, inter-related object properties. To tackle the problem of collecting n-ary semantic knowledge at scale, we propose a transformer neural network that directly generalizes knowledge from observations of object instances. The learned model can reason at different levels of abstraction, effectively predicting unknown properties of objects in different environmental contexts given different amounts of observed information. We quantitatively validate our approach against five prior methods on LINK, a unique dataset we contribute that contains 1457 situated object instances with 15 multimodal properties types and 200 total properties. Compared to the top-performing baseline, a Markov Logic Network, our model obtains a 10% improvement in predicting unknown properties of novel object instances while reducing training and inference time by 150 times. Additionally, we apply our work to a mobile manipulation robot, demonstrating its ability to leverage n-ary reasoning to retrieve objects and actively detect object properties. The code and data are available at https://github.com/wliu88/LINK.

show abstract

Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases

Cited by 59 publications

References 31 publications

A survey on visual transfer learning using knowledge graphs

A survey on visual transfer learning using knowledge graphs

Message Passing for Hyper-Relational Knowledge Graphs

Learning Instance-Level N-Ary Semantic Knowledge At Scale For Robots Operating in Everyday Environments

Contact Info

Product

Resources

About