MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers

Abstract

Developing automatic Math Word Problem (MWP) solvers has been an interest of NLP researchers since the 1960s. Over the last few years, there are a growing number of datasets and deep learning-based methods proposed for effectively solving MWPs. However, most existing methods are benchmarked solely on one or two datasets, varying in different configurations, which leads to a lack of unified, standardized, fair, and comprehensive comparison between methods. This paper presents MWPToolkit, the first open-source framework for solving MWPs. In MWPToolkit, we decompose the procedure of existing MWP solvers into multiple core components and decouple their models into highly reusable modules. We also provide a hyper-parameter search function to boost the performance. In total, we implement and compare 17 MWP solvers on 4 widely-used single equation generation benchmarks and 2 multiple equations generation benchmarks. These features enable our MWPToolkit to be suitable for researchers to reproduce advanced baseline models and develop new MWP solvers quickly. Code and documents are available at https: //github.com/LYH-YF/MWPToolkit.

Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction

“…GroupAttn (Li et al, 2019) 21.5 BERT-BERT (Lan et al, 2021) 24.8 Roberta-Roberta (Lan et al, 2021) 30.3 S2T/G2T GTS * (Xie and Sun, 2019) 30.8 Graph2Tree 36.5 BERT-Tree 32.4 Roberta-GTS (Patel et al, 2021) 41.0 Roberta-Graph2Tree (Patel et al, 2021) 43 from baselines and our models, the choice of encoder appear to be important for solving questions in SVAMP -the results on using Roberta as the encoder are particularly striking. Our best variant ROBERTA-DEDUCTREASONER achieves an accuracy score of 47.3 and is able to outperfrom the best baseline (Roberta-Graph2Tree) by 3.5 points (p < 0.01).…”

“…and Li et al (2020) adopted a graph-to-tree approach to model the quantity relations using the graph convolutional networks (GCN) (Kipf and Welling, 2017). Applying pre-trained language models such as BERT (Devlin et al, 2019) was shown to significantly benefit the tree expression generation (Lan et al, 2021;Tan et al, 2021;Shen et al, 2021). Different from the tree-based generation models, our work is related to deductive systems (Shieber et al, 1995;Nederhof, 2003) where we aim to obtain step-by-step expressions.…”

“…GroupAttn (Li et al, 2019) 69.5 66.9 mBERT-LSTM (Tan et al, 2021) 75.1 -BERT-BERT (Lan et al, 2021) -76.6 Roberta-Roberta (Lan et al, 2021) -76.9…”

“…BERT with an LSTM decoder (mBERT-LSTM). Lan et al (2021) presented two seq2seq models that use BERT/Roberta as both encoder and decoder, namely, BERT-BERT and Roberta-Roberta.…”

The Computational Complexity of Finding Arithmetic Expressions With and Without Parentheses

“…many natural language processing tasks, it was shown that transformers and other large language models like GPT do not perform well on math word problems and mathematical reasoning [6,7] A number of systems have used sequenceto-tree and graph-to-tree neural networks generate arithmetic expression trees from input questions and used those to calculate answers. The main targets have been the Math23k and MAWPS data sets which contain more than 23,000 math word problems [8,11,19,22,12,21,15,10]. Similar techniques have been used to tackle more advanced problems.…”