HMS: A Hierarchical Solver with Dependency-Enhanced Understanding for Math Word Problem

Abstract

Automatically solving math word problems is a crucial task for exploring the intelligence levels of machines in the general AI domain. It is highly challenging since it requires not only natural language understanding but also mathematical expression inference. Existing solutions usually explore sequence-to-sequence models to generate expressions, where the problems are simply encoded sequentially. However, such models are generally far from enough for understanding problems as similar to humans and lead to incorrect answers. To this end, in this paper, we propose a novel Hierarchical Math Solver (HMS) to make deep understanding and exploitation of problems. In problem understanding, imitating human reading habits, we propose a hierarchical word-clause-problem encoder. Specifically, we first split each problem into several clauses and learn problem semantics from the local clause level to the global problem level. Then, in clause understanding, we propose a dependency-based module to enhance clause semantics with the dependency structure of the problem. Next, in expression inference, we propose a novel tree-based decoder to generate the mathematical expression for the answer. In the decoder, we apply a hierarchical attention mechanism to enhance the problem semantics with context from different levels, and a pointer-generator network to guide the model to copy existing information and infer extra knowledge. Extensive experimental results on two widely used datasets demonstrate that HMS achieves not only better answers but also more reasonable inference.

Goal selection and feedback for solving math word problems

“…Zhang et al proposed a novel graph-based encoder to learn the quantity-related features for enhancing problem understanding [5]. Imitating human reading habits, Lin et al proposed a hierarchical word-clause-problem encoder and applied a hierarchical attention mechanism to enhance the problem semantics with context from different levels, and a pointer-generator network to guide the model to copy existing information and infer extra knowledge in decoding [6].…”

“…• HMS [6]: The MWP solver with a dependency-based module for encoding and an improved GTS decoder. • TM-generation [32]: The MWP solver uses the decoder of Transformer to predict math expression templates, and then fills the missing operators in the predicted templates by the operator identification layer they designed.…”

“…In the 1960s, some researchers began to adopt Artificial Intelligence (AI) methods to solve MWPs. Recently, Wang et al used a modified Seq2Seq model to build the MWPs Solver [1] and more and more researchers proposed a variety of approaches based on deep learning for solving MWPs [2][3][4][5][6]. Deep learning-based methods turn MWP solving into a natural language generation task, where the generated text is not "Natural", but consists of mathematical symbols.…”

“…Nowadays, most SOTA models use the decoder of Goal-Driven Tree-structured MWPs Solver (GTS) proposed by Xie and Sun [4] for model decoding and token prediction [5][6][7]. The goal-driven mechanism in human problem solving is proposed by GTS.…”

A Novel Approach for Auto-Formulation of Optimization Problems

“…Many operation research studies have achieved impressive performances in various scenarios, such as bike-share management [1,9] and revenue maximization [2]. To solve the optimization problem automatically, models should have the capability of understanding the natural mathematical problem and generating the mathematical formulation, which has been widely studied [12,7,5,8].…”