A Survey of Knowledge-enhanced Text Generation

Abstract

The goal of text-to-text generation is to make machines express like a human in many applications such as conversation, summarization, and translation. It is one of the most important yet challenging tasks in natural language processing (NLP). Various neural encoder-decoder models have been proposed to achieve the goal by learning to map input text to output text. However, the input text alone often provides limited knowledge to generate the desired output, so the performance of text generation is still far from satisfaction in many real-world scenarios. To address this issue, researchers have considered incorporating (i) internal knowledge embedded in the input text and (ii) external knowledge from outside sources such as knowledge base and knowledge graph into the text generation system. This research topic is known as knowledge-enhanced text generation . In this survey, we present a comprehensive review of the research on this topic over the past five years. The main content includes two parts: (i) general methods and architectures for integrating knowledge into text generation; (ii) specific techniques and applications according to different forms of knowledge data. This survey can have broad audiences, researchers and practitioners, in academia and industry.

Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts

“…Besides, leveraging knowledge graph is not the only way to promote content diversity as it is a highly knowledge-intensive task. Many existing knowledge-enhanced methods (Yu et al, 2022c) can be used to acquire different external knowledge for producing diverse outputs, e.g., taking different retrieved documents as conditions for generator.…”

Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts

“…However, these methods were not able to explicitly control varying semantics units and produce outputs of diverse content. Meanwhile, the input text alone contains too limited knowledge to support diverse reasoning and produce multiple reasonable outputs (Yu et al, 2022c). As an example, Table 1 shows the human evaluation results on two GCR tasks.…”

“…Incorporating external knowledge is essential for many NLG tasks to augment the limited textual information (Yu et al, 2022c;Dong et al, 2021;Yu et al, 2022b). Some recent work explored using graph neural networks (GNN) to reason over multihop relational knowledge graph (KG) paths (Zhou et al, 2018;Jiang et al, 2019;Zhang et al, 2020a;Wu et al, 2020;Yu et al, 2022a;Zeng et al, 2021).…”

UnifieR: A Unified Retriever for Large-Scale Retrieval

“…first-stage retrieval, aims to efficiently fetch all relevant documents for a given query from a large-scale collection with millions or billions of entries 2 . It plays indispensable roles as prerequisites for a broad spectrum of downstream tasks, e.g., information retrieval (IR) [2], open-domain question answering (ODQA) [3], knowledge-grounded conversation (KGC) [63], and recommendation system [67]. To make online large-scale retrieval possible, the common practice is to represent queries and documents by an encoder in a Siamese manner (i.e., Bi-Encoder, BE) [48].…”