RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding

Ji, Ziwei; Liu, Zihan; Lee, Nayeon; Yu, Tiezheng; Wilie, Bryan; Zeng, Min; Fung, Pascale

doi:10.18653/v1/2023.findings-acl.275

Cited by 9 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hallucination is a prevalent problem in generative AI, especially in contexts where the accuracy of information is critical such as misinformation and disinformation on social media. Much research in recent years has been dedicated to its detection [178] and mitigation [179], [180]. Unexpectedly, an alternate perspective has emerged that notes AI hallucination as non-harmful in special cases [160], as even hallucinating LLMs can be used collaboratively as partners to provide research leads, support creative writing, and alleviate writer's block.…”

Section: B Mitigating Hallucinationmentioning

confidence: 99%

Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords

Shah,

Thapa,

Acharya

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This paper explores the dual role of Large Language Models (LLMs) in the context of online misinformation and disinformation. In today's digital landscape, where the internet and social media facilitate the rapid dissemination of information, discerning between accurate content and falsified information presents a formidable challenge. Misinformation, often arising unintentionally, and disinformation, crafted deliberately, are at the forefront of this challenge. LLMs such as OpenAI's GPT-4, equipped with advanced language generation abilities, present a double-edged sword in this scenario. While they hold promise in combating misinformation by fact-checking and detecting LLM-generated text, their ability to generate realistic, contextually relevant text also poses risks for creating and propagating misinformation. Further, LLMs are plagued with many problems such as biases, knowledge cutoffs, and hallucinations, which may further perpetuate misinformation and disinformation. The paper outlines historical developments in misinformation detection and how it affects social media consumption, especially among youth, and introduces LLMs and their applications in various domains. It then critically analyzes the potential of LLMs to generate and counter misinformation and disinformation in sensitive topics such as healthcare, COVID-19, and political agendas. Further, it discusses mitigation strategies, ethical considerations, and regulatory measures, summarizing previous methods and proposing future research direction toward leveraging the benefits of LLMs while minimizing misuse risks. The paper concludes by acknowledging LLMs as powerful tools with significant implications in both spreading and combating misinformation in the digital age.

show abstract

Section: B Mitigating Hallucinationmentioning

confidence: 99%

Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords

Shah,

Thapa,

Acharya

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Measuring Hallucinations Hallucination of language models or the generation of contents that are either non-factual or not supported by evidence have been studied and reported in various fields (Ji et al, 2023b;Bang et al, 2023), such as machine translation (Raunak et al, 2021), abstractive summarization (Maynez et al, 2020;Lee et al, 2022), Open Domain Dialogue (Ji et al, 2023c;, Question Answering (Lin et al, 2022), or image captioning (Rohrbach et al, 2018). Recently developed LLMs such as Bing Chat, or perplexity.ai even serve as generative search engines, although their seemingly fluent and informative responses are not always verifiable (Liu et al, 2023b).…”

Section: Related Workmentioning

confidence: 99%

KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection

Choi,

Fang,

Wang

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Large Language Models (LLMs) have demonstrated remarkable human-level natural language generation capabilities. However, their potential to generate misinformation, often called the hallucination problem, poses a significant risk to their deployment. A common approach to address this issue is to retrieve relevant knowledge and fine-tune the LLM with the knowledge in its input. Unfortunately, this method incurs high training costs and may cause catastrophic forgetting for multitasking models. To overcome these limitations, we propose a knowledge-constrained decoding method called KCTS (Knowledge-Constrained Tree Search), which guides a frozen LM to generate text aligned with the reference knowledge at each decoding step using a knowledge classifier score and MCTS (Monte-Carlo Tree Search). To adapt the sequence-level knowledge classifier to token-level guidance, we also propose a novel token-level hallucination detection method called RIPA (Reward Inflection Point Approximation). Our empirical results on knowledge-grounded dialogue and abstractive summarization demonstrate the strength of KCTS 1 as a plug-and-play, model-agnostic decoding method that can effectively reduce hallucinations in natural language generation.

show abstract

“…In contrast, GenExpan performs better, benefiting from the given contextual corpus and targeted retrieval augmentation strategies. On the other hand, GPT-4 is prone to haphazardly generate non-existent entities (e.g., fake mobile phone brands), which is referred to hallucination problem in recent work [8,45]. We are currently unable to solve this issue by simple output post-processing.…”

Section: Methods Type Methodsmentioning

confidence: 99%

Building Chinese interdisciplinary research centers : the case of Tsinghua University

Li¹,

李蓉暉²

View full text Add to dashboard Cite

Graph machine learning has been extensively studied in both academia and industry. However, in the literature, most existing graph machine learning models are designed to conduct training with data samples in a random order, which may suffer from suboptimal performance due to ignoring the importance of different graph data samples and their training orders for the model optimization status. To tackle this critical problem, curriculum graph machine learning (Graph CL), which integrates the strength of graph machine learning and curriculum learning, arises and attracts an increasing amount of attention from the research community. Therefore, in this paper, we comprehensively overview approaches on Graph CL and present a detailed survey of recent advances in this direction. Specifically, we first discuss the key challenges of Graph CL and provide its formal problem definition. Then, we categorize and summarize existing methods into three classes based on three kinds of graph machine learning tasks, i.e., node-level, linklevel, and graph-level tasks. Finally, we share our thoughts on future research directions. To the best of our knowledge, this paper is the first survey for curriculum graph machine learning.

show abstract

RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding

Cited by 9 publications

References 0 publications

Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords

Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords

KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection

Building Chinese interdisciplinary research centers : the case of Tsinghua University

Contact Info

Product

Resources

About