MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving

Liang, Zhenwen; Zhang, Jipeng; Wang, Lei; Wei, Qin; Lan, Yunshi; Shao, Jie; Zhang, Xiangliang

doi:10.48550/arxiv.2107.13435

Cited by 4 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GenBERT [47] and NF-NSM [39] enhance the numerical reasoning capabilities of models by incorporating numerical data into the training process of PLMs. MWP-BERT [95] further enhances the model's Mathematical Language Models: A Survey 111:9 capacity to represent and calculate numerical values by incorporating numeric attributes into symbol placeholders. MathBERT [136] employs additional joint training of text and formulas to effectively capture the semantic-level structural information of formulas.…”

Section: Non-autoregression Lmsmentioning

confidence: 99%

Geography of Technology Transfer in China

Liu

2023

East China Normal University Scientific Reports

View full text Add to dashboard Cite

Graphs can model complex relationships between objects, enabling a myriad of Web applications such as online page/article classification and social recommendation. While graph neural networks (GNNs) have emerged as a powerful tool for graph representation learning, in an end-to-end supervised setting, their performance heavily relies on a large amount of task-specific supervision. To reduce labeling requirement, the "pre-train, fine-tune" and "pre-train, prompt" paradigms have become increasingly common. In particular, prompting is a popular alternative to fine-tuning in natural language processing, which is designed to narrow the gap between pre-training and downstream objectives in a task-specific manner. However, existing study of prompting on graphs is still limited, lacking a universal treatment to appeal to different downstream tasks. In this paper, we propose GraphPrompt, a novel pre-training and prompting framework on graphs. GraphPrompt not only unifies pre-training and downstream tasks into a common task template, but also employs a learnable prompt to assist a downstream task in locating the most relevant knowledge from the pre-trained model in a task-specific manner. Finally, we conduct extensive experiments on five public datasets to evaluate and analyze GraphPrompt. CCS CONCEPTS• Computing methodologies → Learning latent representations; • Information systems → Data mining.

show abstract

Section: Non-autoregression Lmsmentioning

confidence: 99%

Geography of Technology Transfer in China

Liu

2023

East China Normal University Scientific Reports

View full text Add to dashboard Cite

show abstract

“…Instead, these models can undergo fine-tuning directly on the pre-trained counterparts, streamlining the training process and yielding outstanding results [17,18]. In the work of MWp-bert [19], pre-trained models were employed for solving mathematical word problems. GeoQA+ [20] utilized pre-trained models to transform the textual information of geometric problems, thereby augmenting the dataset.…”

Section: Pre-training Model In Nlpmentioning

confidence: 99%

FGeo-TP: A Language Model-Enhanced Solver for Euclidean Geometry Problems

He,

Zou,

Zhang

et al. 2024

Symmetry

View full text Add to dashboard Cite

The application of contemporary artificial intelligence techniques to address geometric problems and automated deductive proofs has always been a grand challenge to the interdisciplinary field of mathematics and artificial intelligence. This is the fourth article in a series of our works, in our previous work, we established a geometric formalized system known as FormalGeo. Moreover, we annotated approximately 7000 geometric problems, forming the FormalGeo7k dataset. Despite the fact that FGPS (Formal Geometry Problem Solver) can achieve interpretable algebraic equation solving and human-like deductive reasoning, it often experiences timeouts due to the complexity of the search strategy. In this paper, we introduced FGeo-TP (theorem predictor), which utilizes the language model to predict the theorem sequences for solving geometry problems. The encoder and decoder components in the transformer architecture naturally establish a mapping between the sequences and embedding vectors, exhibiting inherent symmetry. We compare the effectiveness of various transformer architectures, such as BART or T5, in theorem prediction, and implement pruning in the search process of FGPS, thereby improving its performance when solving geometry problems. Our results demonstrate a significant increase in the problem-solving rate of the language model-enhanced FGeo-TP on the FormalGeo7k dataset, rising from 39.7% to 80.86%. Furthermore, FGeo-TP exhibits notable reductions in solution times and search steps across problems of varying difficulty levels.

show abstract

“…Previous work has seen modest success on simpler or specialized mathematics problem benchmarks. Techniques based on co-training output to verify (7,8) or predict expression trees to evaluate (9)(10)(11)(12)(13)(14) are able to solve elementary schoollevel math problems, such as MAWPS and Math23k, with over 80% accuracy. However, these approaches does not extend to high-school, math Olympiad, nor university-level courses.…”

Section: Related Workmentioning

confidence: 99%

“…The parabola y = a + bx + cx 2 goes through the points (x, y) = (1, 4) and (2, 8) and (3,14). Find and solve a matrix equation for the unknowns (a, b, c).…”

Section: Topic Elimination Using Matrices Original Questionmentioning

confidence: 99%

A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level

Drori,

Zhang,

Shuttleworth

et al. 2021

Preprint

View full text Add to dashboard Cite

We demonstrate that a neural network pre-trained on text and finetuned on code solves Mathematics problems by program synthesis. We turn questions into programming tasks, automatically generate programs, and then execute them, perfectly solving university-level problems from MIT's large Mathematics courses (Single Variable Calculus 18.01, Multivariable Calculus 18.02, Differential Equations 18.03, Introduction to Probability and Statistics 18.05, Linear Algebra 18.06, and Mathematics for Computer Science 6.042), Columbia University's COMS3251 Computational Linear Algebra course, as well as questions from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Number Theory, and Precalculus), the latest benchmark of advanced mathematics problems specifically designed to assess mathematical reasoning. We explore prompt generation methods that enable Transformers to generate question solving programs for these subjects, including solutions with plots. We generate correct answers for a random sample of questions in each topic. We quantify the gap between the original and transformed questions and perform a survey to evaluate the quality and difficulty of generated questions. This is the first work to automatically solve, grade, and generate university-level Mathematics course questions at scale. This represents a milestone for higher education. Neural networks | Mathematics courses | Answering and generating questions |

show abstract

MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving

Cited by 4 publications

References 20 publications

Geography of Technology Transfer in China

Geography of Technology Transfer in China

FGeo-TP: A Language Model-Enhanced Solver for Euclidean Geometry Problems

A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level

Contact Info

Product

Resources

About