Schema Retrieval for Korean Geographic Knowledge Base Question Answering Using Few-Shot Prompting

Abstract

Geographic Knowledge Base Question Answering (GeoKBQA) has garnered increasing attention for its ability to process complex geographic queries. This study focuses on schema retrieval, a critical step in GeoKBQA that involves extracting relevant schema items (classes, relations, and properties) to generate accurate operational queries. Current GeoKBQA studies primarily rely on rule-based approaches for schema retrieval. These predefine words or descriptions for each schema item. This rule-based method has three critical limitations: (1) poor generalization to undefined schema items, (2) failure to consider the semantic meaning of user queries, and (3) an inability to adapt to languages not used in the predefined step. In this study, we present a schema retrieval model by using few-shot prompting on GPT-4 Turbo to address these issues. Using the SKRE dataset, we searched for the best prompt in terms of enabling the model to handle Korean geographic questions across various generalization levels. Notably, this method outperformed fine-tuning in zero-shot scenarios, underscoring its adaptability to unseen data. To our knowledge, this is the first attempt to develop a schema retrieval model for GeoKBQA that purely utilizes a language model and is capable of processing Korean geographic questions.