Injecting Numerical Reasoning Skills into Knowledge Base Question Answering Models

Abstract

Embedding-based methods are popular for Knowledge Base Question Answering (KBQA), but few current models have numerical reasoning skills and thus struggle to answer ordinal constrained questions. This paper proposes a new embedding-based KBQA framework which particularly takes numerical reasoning into account. We present NumericalTransformer on top of NSM, a state-of-the-art embedding-based KBQA model, to create NT-NSM. To enable better training, we propose two pre-training tasks with explicit numerical-oriented loss functions on two generated training datasets and a template-based data augmentation method for enriching ordinal constrained QA dataset. Extensive experiments on KBQA benchmarks demonstrate that with the help of our training algorithm, NT-NSM is empowered with numerical reasoning skills and substantially outperforms the baselines in answering ordinal constrained questions.

Geography of Technology Transfer in China

“…For example, Aristo [26] fine-tunes BERT using scientific curriculum data, yielding promising results on science exams. GenBERT [47] and NF-NSM [39] enhance the numerical reasoning capabilities of models by incorporating numerical data into the training process of PLMs. MWP-BERT [95] further enhances the model's Mathematical Language Models: A Survey 111:9 capacity to represent and calculate numerical values by incorporating numeric attributes into symbol placeholders.…”