Awakening Latent Grounding from Pretrained Language Models for Semantic Parsing

Liu, Qian; Yang, Dejian; Zhang, Jiahui; Guo, Jiaqi; Zhou, Bin; Lou, Jian–Guang

doi:10.18653/v1/2021.findings-acl.100

Cited by 15 publications

(13 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the (Lei et al 2020), we also report the result of SLSQL + BERT (Oracle), where the learnable schema linking module is replaced with human annotations in inference. It represents the maximum potential benefit of schema linking for the text-to-SQL task (Liu et al 2021). As mentioned, when introducing SDCUP, it dramatically improves the performance, up to almost 14.4% by base model.…”

Section: Experiments Results and Analysesmentioning

confidence: 94%

Linking-Enhanced Pre-Training for Table Semantic Parsing

Qin¹,

Wang²,

Hui³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently pre-training models have significantly improved the performance of various NLP tasks by leveraging large-scale text corpora to improve the contextual representation ability of the neural network. The large pre-training language model has also been applied in the area of table semantic parsing. However, existing pre-training approaches have not carefully explored explicit interaction relationships between a question and the corresponding database schema, which is a key ingredient for uncovering their semantic and structural correspondence. Furthermore, the question-aware representation learning in the schema grounding context has received less attention in pre-training objective. To alleviate these issues, this paper designs two novel pre-training objectives to impose the desired inductive bias into the learned representations for table pre-training. We further propose a schema-aware curriculum learning approach to mitigate the impact of noise and learn effectively from the pre-training data in an easyto-hard manner. We evaluate our pre-trained framework by fine-tuning it on two benchmarks, Spider and SQUALL. The results demonstrate the effectiveness of our pre-training objective and curriculum compared to a variety of baselines. * Equal contribution. † Work done during an internship at Alibaba Group.

show abstract

Section: Experiments Results and Analysesmentioning

confidence: 94%

Linking-Enhanced Pre-Training for Table Semantic Parsing

Qin¹,

Wang²,

Hui³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Following the previous works [25,30], we report the micro average precision, recall and F1-score for both columns (𝐶𝑜𝑙 𝑃 , 𝐶𝑜𝑙 𝑅 , 𝐶𝑜𝑙 𝐹 ) and tables (𝑇 𝑎𝑏 𝑃 ,𝑇 𝑎𝑏 𝑅 , 𝑇 𝑎𝑏 𝐹 ). The metric focus on whether the correct schema item is identified.…”

Section: Schema Linking Analysismentioning

confidence: 99%

“…(4) SLSQL 𝐿 [25] is trained with full schema linking supervision by a learnable schema linking module. ( 5) ETA [30] trains the schema linking module using pseudo alignment as supervision generated from PLMs.…”

Section: Schema Linking Analysismentioning

confidence: 99%

“…Dong et al [11] leverage the implicit supervision from SQL queries to guide the schema linking training by reinforcement learning, which ignores the semantic information and may still fail in synonym substitution settings. To better capture the semantic relationship between question and schema, Liu et al [30] train the schema linking module under the supervision of pseudo alignment between token and schema items from PLMs. However, the supervision from PLMs could be noisy or incomplete.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graph

Liu

et al. 2022

Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

The generalizability to new databases is of vital importance to Text-to-SQL systems which aim to parse human utterances into SQL statements. Existing works achieve this goal by leveraging the exact matching method to identify the lexical matching between the question words and the schema items. However, these methods fail in other challenging scenarios, such as the synonym substitution in which the surface form differs between the corresponding question words and schema items. In this paper, we propose a framework named ISESL-SQL to iteratively build a semantic enhanced schema-linking graph between question tokens and database schemas. First, we extract a schema linking graph from PLMs through a probing procedure in an unsupervised manner. Then the schema linking graph is further optimized during the training process through a deep graph learning method. Meanwhile, we also design an auxiliary task called graph regularization to improve the schema information mentioned in the schema-linking graph. Extensive experiments on three benchmarks demonstrate that ISESL-SQL could consistently outperform the baselines and further investigations show its generalizability and robustness.

show abstract

“…To capture such alignments, several attention-based models were proposed (Shi et al, 2020;Lei et al, 2020;Liu et al, 2021), which employ the attention weights among tokens to indicate the alignments. Specifically, they use an attention module to perform schema linking at the encoding stage (Lei et al, 2020;Liu et al, 2021), and may use another attention to align each output token to its corresponding input tokens at the decoding stage (Shi et al, 2020). However, we argue that the attention mechanism is not an appropriate way to capture and leverage lexico-logical alignments.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Explicit Lexico-logical Alignments in Text-to-SQL Parsing

Sun¹,

He²,

Zhu³

et al. 2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

Text-to-SQL aims to parse natural language questions into SQL queries, which is valuable in providing an easy interface to access large databases. Previous work has observed that leveraging lexico-logical alignments is very helpful to improve parsing performance. However, current attention-based approaches can only model such alignments at the token level and have unsatisfactory generalization capability. In this paper, we propose a new approach to leveraging explicit lexico-logical alignments. It first identifies possible phrase-level alignments and injects them as additional contexts to guide the parsing procedure. Experimental results on SQUALL show that our approach can make better use of such alignments and obtains an absolute improvement of 3.4% compared with the current state-of-the-art.

show abstract

Awakening Latent Grounding from Pretrained Language Models for Semantic Parsing

Cited by 15 publications

References 41 publications

Linking-Enhanced Pre-Training for Table Semantic Parsing

Linking-Enhanced Pre-Training for Table Semantic Parsing

Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graph

Leveraging Explicit Lexico-logical Alignments in Text-to-SQL Parsing

Contact Info

Product

Resources

About