A Biblioteca Pessoal Do Visconde Do Rio Branco

Pereira, Pedro Henrique Rodrigues; Duarte, Caio Henrique Dias

doi:10.9771/rpa.v16i3.52320

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Initially, we attempted to use the LLM to extract variable name mentions or perform conventional query decomposition like in (Pereira et al, 2022). While these showed promise, we obtained much better results when we took a leap and harnessed the power of an LLM to directly hallucinate a DB schema that could be used to answer the question x.…”

Section: Llm-based Query Transformationmentioning

confidence: 99%

“…LLMs for question decompostion and retrieval: Decomposing complex questions has been of interest for some years now (Wolfson et al, 2020). In the context of LLMs, Pereira et al (2022) propose a question answering system where supporting evidence to answer a question can be spread over multiple documents. Similar to us, they use an LLM to decompose the question using five in-context examples.…”

Section: Retrieval Objectivementioning

confidence: 99%

See 1 more Smart Citation

CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQL

Kothyari,

Dhingra,

Sarawagi

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Existing Text-to-SQL generators require the entire schema to be encoded with the user text. This is expensive or impractical for large databases with tens of thousands of columns. Standard dense retrieval techniques are inadequate for schema subsetting of a large structured database, where the correct semantics of retrieval demands that we rank sets of schema elements rather than individual elements. In response, we propose a two-stage process for effective coverage during retrieval. First, we instruct an LLM to hallucinate a minimal DB schema deemed adequate to answer the query. We use the hallucinated schema to retrieve a subset of the actual schema, by composing the results from multiple dense retrievals. Remarkably, hallucination -generally considered a nuisance -turns out to be actually useful as a bridging mechanism. Since no existing benchmarks exist for schema subsetting on large databases, we introduce three benchmarks. Two semi-synthetic datasets are derived from the union of schemas in two wellknown datasets, SPIDER and BIRD, resulting in 4502 and 798 schema elements respectively. A real-life benchmark called SocialDB is sourced from an actual large data warehouse comprising 17844 schema elements. We show that our method 1 leads to significantly higher recall than SOTA retrieval-based augmentation methods.

show abstract

Section: Llm-based Query Transformationmentioning

confidence: 99%

Section: Retrieval Objectivementioning

confidence: 99%