A Survey on Retrieval-Augmented Text Generation

Li, Huayang; Su, Yixuan; Wang, Yan; Liu, Lemao

doi:10.48550/arxiv.2202.01110

Cited by 8 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our experiments demonstrate that platforms have a variety of ways to prioritize cost vs. accuracy. Strategies include using smaller, less expensive LLMs (e.g., textbison vs. text-unicorn); simplifying policies via automatic prompt optimization [43], [56]; or optimizing the number of examples in a prompt or even the candidate selection strategy [29]. We envision that real-world deployments will consist of distinct layers of LLM raters each with supporting RAG databases.…”

Section: Optimizations and Future Workmentioning

confidence: 99%

SoK: Hate, Harassment, and the Changing Landscape of Online Abuse

Thomas

Akhawe²,

Bailey

et al. 2021

2021 IEEE Symposium on Security and Privacy (SP)

View full text Add to dashboard Cite

In this paper, we explore the feasibility of leveraging large language models (LLMs) to automate or otherwise assist human raters with identifying harmful content including hate speech, harassment, violent extremism, and election misinformation. Using a dataset of 50,000 comments, we demonstrate that LLMs can achieve 90% accuracy when compared to human verdicts. We explore how to best leverage these capabilities, proposing five design patterns that integrate LLMs with human rating, such as pre-filtering non-violative content, detecting potential errors in human rating, or surfacing critical context to support human rating. We outline how to support all of these design patterns using a single, optimized prompt. Beyond these synthetic experiments, we share how piloting our proposed techniques in a real-world review queue yielded a 41.5% improvement in optimizing available human rater capacity, and a 9-11% increase (absolute) in precision and recall for detecting violative content.

show abstract

Section: Optimizations and Future Workmentioning

confidence: 99%

SoK: Hate, Harassment, and the Changing Landscape of Online Abuse

Thomas

Akhawe²,

Bailey

et al. 2021

2021 IEEE Symposium on Security and Privacy (SP)

View full text Add to dashboard Cite

show abstract

“…In generative LLMs, the technique of Retrieval Augmented Generation (RAG) is used to supply external context into the model's input [15] with the goal of augmenting its output quality in terms of factuality. More specifically, the original input is used to retrieve context based on a criterion, usually embedding similarity, from a pool of available factually correct and externally provided texts.…”

Section: Retrieval Augmented Generationmentioning

confidence: 99%

Can Large Language Models Revolutionalize Open Government Data Portals? A Case of Using ChatGPT in statistics.gov.scot

Mamalis,

Kalampokis,

Karamanou

et al. 2023

Preprint

View full text Add to dashboard Cite

Large language models possess tremendous natural language understanding and generation abilities. However, they often lack the ability to discern between fact and fiction, leading to factually incorrect responses. Open Government Data are repositories of, often times linked, information that is freely available to everyone. By combining these two technologies in a proof of concept designed application utilizing the GPT3.5 OpenAI model and the Scottish open statistics portal, we show that not only is it possible to augment the large language model's factuality of responses, but also propose a novel way to effectively access and retrieve statistical information from the data portal just through natural language querying. We anticipate that this paper will trigger a discussion regarding the transformation of Open Government Portals through large language models.

show abstract

“…In CoALA, a retrieval procedure (Li et al, 2022a;Gu et al, 2018) reads information from long-term memories into working memory. Depending on the information and memory type, it could be implemented in various ways, e.g., rule-based, sparse, or dense retrieval.…”

Section: Retrieval Actionsmentioning

confidence: 99%

Show or Tell? Exploring when (and why) teaching with language outperforms demonstration

Sumers¹,

Ho²,

Hawkins³

et al. 2021

Preprint

View full text Add to dashboard Cite

People use a wide range of communicative acts, from concrete demonstrations to abstract language. What are the strengths and weaknesses of such different modalities? We present a series of real-time, multi-player experiments asking participants to teach (Boolean) concepts using either demonstrations or language. Our first experiment (N = 454) manipulated the complexity of the concept, finding that linguistic (but not demonstrative) teaching enables high-fidelity transmission of more complex concepts. Why, then, do humans use both demonstrations and language? As a form of conventionalized communication, language relies on shared context between speaker and listener, whereas demonstrations are inherently grounded in the world. We hypothesized linguistic communication would be more sensitive to perturbations of shared context than demonstrations. Our second experiment (N = 568) manipulated teachers’ ability to see the features that defined the concept. This restriction severely impaired linguistic (but not demonstrative) teaching. Our comparative approach confirms language relies on shared context to permit high bandwidth communication; in contrast, demonstrations are lower-bandwidth but more robust.

show abstract

A Survey on Retrieval-Augmented Text Generation

Cited by 8 publications

References 42 publications

SoK: Hate, Harassment, and the Changing Landscape of Online Abuse

SoK: Hate, Harassment, and the Changing Landscape of Online Abuse

Can Large Language Models Revolutionalize Open Government Data Portals? A Case of Using ChatGPT in statistics.gov.scot

Show or Tell? Exploring when (and why) teaching with language outperforms demonstration

Contact Info

Product

Resources

About