CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases

Chen, Yu; Zhang, Rui; Er, He; Li, Suyi; Xue, Eric; Pang, Bo; Victoria, Lin, Xi; Tan, Yi Chern; Shi, Tingna; Jiang, Youxuan; Yasunaga, Michihiro; Shim, Sungrok; Chen, Tao; Fabbri, Alexander R.; Li, Zifan; Chen, Luyao; Zhang, Kun; Dixit, Shreya; Zhang, Vincent; Xiong, Caiming; Socher, Richard; Lasecki, Walter S.; Radev, Dragomir

doi:10.48550/arxiv.1909.05378

Cited by 5 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, following the methodology used to construct popular NL-to-SQL datasets such as WikiSQL [33], a crowdsourced task could be setup to paraphrase the current set of utterances and then to validate the paraphrased utterances. This paraphrasing can help generate a larger dataset which could subsequently enable more quantitative analyses of utterances and the development of NL-to-Visualization models, complementing research trends in the computer vision (e.g., [3,8]) and database communities (e.g., [31,32]). Lastly, we asked participants to specify only one chart at a time.…”

Section: Limitations and Future Workmentioning

confidence: 99%

Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations

Srinivasan

Nyapathy

Lee

et al. 2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Natural language interfaces (NLIs) for data visualization are becoming increasingly popular both in academic research and in commercial software. Yet, there is a lack of empirical understanding of how people specify visualizations through natural language.We conducted an online study (N = 102), showing participants a series of visualizations and asking them to provide utterances they would pose to generate the displayed charts. From the responses, we curated a dataset of 893 utterances and characterized the utterances according to (1) their phrasing (e.g., commands, queries, questions) and ( 2) the information they contained (e.g., chart types, data aggregations). To help guide future research and development, we contribute this utterance dataset and discuss its applications toward the creation and benchmarking of NLIs for visualization. CCS CONCEPTS• Human-centered computing → Visualization; Empirical studies in visualization; Natural language interfaces.

show abstract

Section: Limitations and Future Workmentioning

confidence: 99%

Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations

Srinivasan

Nyapathy

Lee

et al. 2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…Due to the variety of domains and languages over different datasets, it is difficult to measure all semantic parsing methods in a unified framework. To address this issue, Yu et al (2018), and Yu et al (2019a) built different shared-task platforms with leaderboard for semantic parsing evaluation on the common datasets and consistent evaluation metrics.…”

Section: Discussionmentioning

confidence: 99%

“…Multi-party Scenarios Similar to the scenario of SPARC, to obtain the answers to the questions sampled from SPIDER (Yu et al, 2018), the conversations in COSQL (Yu et al, 2019a) are conducted between two human interlocutors, who play the roles of user and system, respectively. The dialogues in the SPACEBOOK (Vlachos and Clark, 2014) are under the scenarios formed by the routing requests.…”

Section: Scenariosmentioning

confidence: 99%

Section: Context and Annotationsmentioning

confidence: 99%

“…For the datasets, SPACEBOOK and COSQL, Yu et al (2019a) and Vlachos and Clark (2014) label utterances with dialogue acts along with MRs. Different from (Yu et al, 2019a), Vlachos and Clark (2014) integrated the dialogue acts into the MRs. The dialogue acts can be considered as the overall functions of the utterances while different dialogue acts reflect different properties of utterances.…”

Section: Context and Annotationsmentioning

confidence: 99%

See 2 more Smart Citations

Context Dependent Semantic Parsing: A Survey

Haffari

2020

Preprint

View full text Add to dashboard Cite

Semantic parsing is the task of translating natural language utterances into machine-readable meaning representations. Currently, most semantic parsing methods are not able to utilize contextual information (e.g. dialogue and comments history), which has a great potential to boost semantic parsing performance. To address this issue, context dependent semantic parsing has recently drawn a lot of attention. In this survey, we investigate progress on the methods for the context dependent semantic parsing, together with the current datasets and tasks. We then point out open problems and challenges for future research in this area. The collected resources for this topic are available at: https://github.com/zhuang-li/ Contextual-Semantic-Parsing-Paper-List.

show abstract

Neural Approaches to Conversational AI

Gao

Galley

2019

FNT in Information Retrieval

170

View full text Add to dashboard Cite

A conversational information retrieval (CIR) system is an information retrieval (IR) system with a conversational interface which allows users to interact with the system to seek information via multi-turn conversations of natural language, in spoken or written form. Recent progress in deep learning has brought tremendous improvements in natural language processing (NLP) and conversational AI, leading to a plethora of commercial conversational services that allow naturally spoken and typed interaction, increasing the need for more human-centric interactions in IR. As a result, we have witnessed a resurgent interest in developing modern CIR systems in both research communities and industry. This book surveys recent advances in CIR, focusing on neural approaches that have been developed in the last few years. This book is based on the authors' tutorial at SIGIR'2020 [Gao et al., 2020b], with IR and NLP communities as the primary target audience. However, audiences with other background, such as machine learning and human-computer interaction, will also find it an accessible introduction to CIR. We hope that this book will prove a valuable resource for students, researchers, and software developers. This manuscript is a working draft. Comments are welcome.

show abstract

CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases

Cited by 5 publications

References 0 publications

Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations

Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations

Context Dependent Semantic Parsing: A Survey

Neural Approaches to Conversational AI

Contact Info

Product

Resources

About