Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization

Abstract

Despite recent advances in abstractive summarization, current summarization systems still suffer from content hallucinations where models generate text that is either irrelevant or contradictory to the source document. However, prior work has been predicated on the assumption that any generated facts not appearing explicitly in the source are undesired hallucinations. Methods have been proposed to address this scenario by ultimately improving 'faithfulness' to the source document, but in reality, there is a large portion of entities in the gold reference targets that are not directly in the source. In this work, we show that these entities are not aberrations, but they instead require utilizing external world knowledge to infer reasoning paths from entities in the source. We show that by utilizing an external knowledge base, we can improve the faithfulness of summaries without simply making them more extractive, and additionally, we show that external knowledge bases linked from the source can benefit the factuality of generated summaries.

A Read-and-Select Framework for Zero-shot Entity Linking

“…Entity linking (EL) bridges the gap between knowledge and downstream tasks (Wang et al, 2023;Dong et al, 2022;. There have been great achievements in building general EL systems with Wikipedia as the corresponding knowledge base (KB).…”

“…", the mention Indiana Jones should be linked to the entity Indiana Jones (minifigure). As a bridge that connects mentions in unstructured text and entities in structured KBs, EL plays a key role in a variety of tasks, including KB population (Ji and Nothman, 2016), semantic search (Blanco et al, 2015), summarization (Dong et al, 2022), etc.…”