Natural SQL: Making SQL Easier to Infer from Natural Language Specifications

Abstract

Addressing the mismatch between natural language descriptions and the corresponding SQL queries is a key challenge for text-to-SQL translation. To bridge this gap, we propose an SQL intermediate representation (IR) called Natural SQL (NatSQL). Specifically, NatSQL preserves the core functionalities of SQL, while it simplifies the queries as follows: (1) dispensing with operators and keywords such as GROUP BY, HAVING, FROM, JOIN ON, which are usually hard to find counterparts for in the text descriptions; (2) removing the need for nested subqueries and set operators; and (3) making schema linking easier by reducing the required number of schema items. On Spider, a challenging textto-SQL benchmark that contains complex and nested SQL queries, we demonstrate that Nat-SQL outperforms other IRs, and significantly improves the performance of several previous SOTA models. Furthermore, for existing models that do not support executable SQL generation, NatSQL easily enables them to generate executable SQL queries, and achieves the new state-of-the-art execution accuracy 1 .

Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment

“…Table 2 presents the difference between the SQL in Spider and the SQL generated by NatSQL in Spider-SS. Our evaluation results are lower than the original NatSQL dataset (Gan et al, 2021b) because the Spider-SS uses equivalent SQL and corrects some errors, as discussed in Section 2.3. Some equivalent and corrected SQL cannot get positive results in exact match metric and execution match.…”

“…The difficulty criteria are defined by Spider benchmark, including easy, medium, hard and extra hard. Experiments show that the more difficult the SQL is, the more difficult it is to predict correctly Shi et al, 2021;Gan et al, 2021b). It can be found from Table 3 that the difficulty distribution of CG-SUB T and CG-SUB D is similar to that of Spider D .…”

“…Therefore, the model trained on Spider-SS may not be ideal for chasing the Spider benchmark, especially based on the exact match metric. Similarly, the RATSQL G extending Nat-SQL had achieved a previous SOTA result in the execution match of the Spider test set but get a worse result than the original in the exact match (Gan et al, 2021b). Thus, we recommend using NatSQL-based datasets to evaluate models trained on NatSQL.…”

“…Unlike Spider, which annotates a whole SQL query to an entire sentence, Spider-SS annotates the SQL clauses to sub-sentences. Spider-SS uses NatSQL (Gan et al, 2021b) instead of SQL for annotation, because it is sometimes difficult to annotate the sub-sentences with corresponding SQL clauses due to the SQL language design. The Spider-SS provides a combination algorithm that collects all NatSQL clauses and then generates the NatSQL query, where the NatSQL query can be converted into an SQL query.…”

“…Next, we annotate every sub-sentence with its cor- , and how much does it weigh? responding SQL clause, reducing the difficulty of this task by using the intermediate representation language NatSQL (Gan et al, 2021b), which is simpler and syntactically aligns better with natural language (NL). Spider-SS thus provides a new resource for designing models with better generalization capabilities without designing a complex alignment algorithm.…”