From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer

Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph Completion

“…These models learn structural information of a knowledge graph, regardless of the textual information of the entities and relations. Recently, Yao et al (2019); Wang et al (2021a); Xie et al (2022b); Saxena et al (2022) proposed to encode entity and relation textual knowledge into the model by using PLMs. Instead of calculating scores from embeddings, they train PLMs to produce plausibility scores for KG text representation.…”

“…Firstly, to remedy the KG structure information loss caused by the naïve "text-to-text" format, we improve KG-S2S via 1) the input representations of entities and relations using Entity Description, Soft Prompt and Seq2Seq Dropout; 2) the constrained inference algorithm empowered by the Prefix Constraints; Secondly, we treat all the KG elements (i.e., entity, relation and timestamp) as "flat" text (Figure 1) which enables KG-S2S to i) handle various verbalizable knowledge graph structures; ii) generate non-entity text and find novel entities for KGs. We make several improvements on the preliminary attempts of concurrent works (Saxena et al, 2022;Xie et al, 2022b) using Seq2Seq for KGC. Our model adds special treatments to input entity/relation textual representation.…”

PromptKG: A Prompt Learning Framework for Knowledge Graph Representation Learning and Application

“…To guarantee the consistency of decoding sequential schemas and tokens in KG, GenKGC [14] proposes an entity-aware hierarchical decoder to constrain 𝑋 𝑡 . In addition, inspired by prompt-learning, GenKGC takes triples with the same relation as demonstrations to implicitly encode structured knowledge.…”

“…Comparing to its high time complexity, StAR [12] and SimKGC [13] both introduce a tower-based method to precompute entity embeddings and retrieve top-𝑘 entities efficiently. Further, GenKGC [14] and KGT5 [9] treat knowledge graph completion as sequence-to-sequence generation. Besides, 𝑘NN-KGE [18] is a KG representation model which linearly interpolates its entity distribution by k-nearest neighbors.…”

“…By comparison, text-based methods [15] incorporate available texts for knowledge representation learning. With the development of prompt learning, lots of text-based models [9,14] have been proposed, which can obtain promising performance with pre-trained language models (PLMs) and take advantage of allocating a fixed memory footprint for those large-scale real-world KGs. However, there is no comprehensive open-sourced framework particularly designed for KG representation with prompt learning at present, Figure 2: A prompt learning method for KG representation learning in PromptKG and those KG can be applied to tasks such as KGC, QA, recommendation and knowledge probing.…”