RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

Wang, Bailin; Shin, Richard; Liu, Xiaodong; Polozov, Oleksandr; Richardson, Matthew

doi:10.48550/arxiv.1911.04942

Cited by 30 publications

(66 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…P e (y|x t , i t ) = ∏ j=1..m P dec (y j |E(x t ), i, y <j ), but additionally uses i, the position of the non-terminal being expanded. We implement P dec (y j |E(x t ), i t , y <j ) with a (causal) relational transformer decoder, similar to Wang et al [36]. Relational transformers augment the attention mechanism by incorporating predefined relationships among elements.…”

Section: Grammformermentioning

confidence: 99%

Learning to Complete Code with Sketches

Guo¹,

Svyatkovskiy²,

Yin³

et al. 2021

Preprint

View full text Add to dashboard Cite

Traditional generative models are limited to predicting sequences of terminal tokens. However, ambiguities in the generation task may lead to incorrect outputs. Towards addressing this, we introduce GRAMMFORMERs, transformer-based grammarguided models that learn (without explicit supervision) to generate sketches -sequences of tokens with holes. Through reinforcement learning, GRAMMFORMERs learn to introduce holes avoiding the generation of incorrect tokens where there is ambiguity in the target task. We train GRAMMFORMERs for statement-level source code completion, i.e. the generation of code snippets given an ambiguous user intent, such as a partial code context. We evaluate GRAMMFORMERs on code completion for C# and Python and show that it generates 10-50% more accurate sketches compared to traditional generative models and 37-50% longer sketches compared to sketch-generating baselines trained with similar techniques.Preprint. Under review.

show abstract

Section: Grammformermentioning

confidence: 99%

Learning to Complete Code with Sketches

Guo¹,

Svyatkovskiy²,

Yin³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For a multi-head attention layer, we replicate the update in equation 2 across all heads. With the aid of ground truth relations, (b ) has been used to modify the attention in the GREAT [29] and RAT-SQL [73] models, whereas the combination of (b ) and (d ) has been used in the Code Transformer model [84]. Since the edges we model are sparse, the additional term in equation 2 can be computed and backpropagated through with sparse primitives in standard automatic differentiation libraries.…”

Section: Using Predicted Relationsmentioning

confidence: 99%

“…Many machine learning models for code take into account relational structure. These models include variants of graph neural networks [3] and Transformers using relative positions [65,29,73,84]. Our focus is specifically on models that modify the standard attention computation [65,19] by altering q or k using edge embeddings.…”

Section: Introductionmentioning

confidence: 99%

Learning to Extend Program Graphs to Work-in-Progress Code

Li,

Maddison,

Tarlow

2021

Preprint

View full text Add to dashboard Cite

Source code spends most of its time in a broken or incomplete state during software development. This presents a challenge to machine learning for code, since highperforming models typically rely on graph structured representations of programs derived from traditional program analyses. Such analyses may be undefined for broken or incomplete code. We extend the notion of program graphs to workin-progress code by learning to predict edge relations between tokens, training on well-formed code before transferring to work-in-progress code. We consider the tasks of code completion and localizing and repairing variable misuse in a work-in-process scenario. We demonstrate that training relation-aware models with fine-tuned edges consistently leads to improved performance on both tasks.

show abstract

“…Global gated graph neural network (Bogin et al, 2019) is designed to train the structure of database patterns and apply it in the encoding and decoding stages. Recently RAT-SQL (Wang et al, 2019) uses a relation-aware self-attention mechanism for schema encoding, feature representation and schema linking. It obtains the state-of-art accuracy of 65.6 on Spider test set.…”

Section: Related Workmentioning

confidence: 99%

GP: Context-free Grammar Pre-training for Text-to-SQL Parsers

Zhao,

Cao,

Zhao

2021

Preprint

View full text Add to dashboard Cite

A new method for Text-to-SQL parsing, Grammar Pre-training (GP), is proposed to decode deep relations between question and database. Firstly, to better utilize the information of databases, a random value is added behind a question word which is recognized as a column, and the new sentence serves as the model input. Secondly, initialization of vectors for decoder part is optimized, with reference to the former encoding so that question information can be concerned. Finally, a new approach called flooding level is adopted to get the non-zero training loss which can generalize better results. By encoding the sentence with GRAPPA and RAT-SQL model, we achieve better performance on spider, a cross-DB 69.8 test). Experiments show that our method is easier to converge during training and has excellent robustness.

show abstract

RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

Cited by 30 publications

References 15 publications

Learning to Complete Code with Sketches

Learning to Complete Code with Sketches

Learning to Extend Program Graphs to Work-in-Progress Code

GP: Context-free Grammar Pre-training for Text-to-SQL Parsers

Contact Info

Product

Resources

About