Reasoning about Quantities in Natural Language

Roy, Subhro; Vieira, Tim; Roth, Dan

doi:10.1162/tacl_a_00118

Cited by 109 publications

(77 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The operator '=' is used for most verbs. For the verb 'to be', we rely on Roy et al [103] which, for example, identify the operator '>' for "capacitance is above 600". More precisely, we apply Roy's approach on the phrase tagged with the support-role used to identify the rhs-term.…”

Section: Identification Of the Ocl Operatorsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic generation of system test cases from use case specifications

Wang

Pastore

Göknil

et al. 2015

Proceedings of the 2015 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

In safety critical domains, system test cases are often derived from functional requirements in natural language (NL) and traceability between requirements and their corresponding test cases is usually mandatory. The definition of test cases is therefore time-consuming and error prone, especially so given the quickly rising complexity of embedded systems in many critical domains. Though considerable research has been devoted to automatic generation of system test cases from NL requirements, most of the proposed approaches require significant manual intervention or additional, complex behavioral modelling. This significantly hinders their applicability in practice.In this paper, we propose Use Case Modelling for System Tests Generation (UMTG), an approach that automatically generates executable system test cases from use case specifications and a domain model, the latter including a class diagram and constraints. Our rationale and motivation are that, in many environments, including that of our industry partner in the reported case study, both use case specifications and domain modelling are common and accepted practice, whereas behavioural modelling is considered a difficult and expensive exercise if it is to be complete and precise. In order to extract behavioral information from use cases and enable test automation, UMTG employs Natural Language Processing (NLP), a restricted form of use case specifications, and constraint solving.

show abstract

Section: Identification Of the Ocl Operatorsmentioning

confidence: 99%

“…For phrases with a quantifier referring to a certain number of members of a group (e.g., at least five), we generate expressions following the COUNT sub-pattern. We also rely on Roy's approach [103] to generate a quantifier formula with an operator and a numeric literal.…”

Section: Identification Of the Type Of Ocl Query Elementmentioning

confidence: 99%

Automatic generation of system test cases from use case specifications

Wang

Pastore

Göknil

et al. 2015

Proceedings of the 2015 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…As to applying coreference resolvers in MWP sovers, the Illinois Coreference Resolver [87] [89] is used in [24] to identify pronoun referents and facilitate semantic labeling. Alternatively, a rule function Coref (A, B), which is true when A and B represent the same entity, is derived in [83] as a component of the declarative rules to determine the math operators.…”

Section: Coreference Resolutionmentioning

confidence: 99%

“…Number of quantities mentioned in text [30] [32] [17] Unigrams and bigrams of sentences in the problem text [24] [50] a sentence "2 footballs and 3 soccer balls cost 220 dollars", the dependency paths between two quantity pairs (2, 220) and (3,220) are identical, implying that 2 and 3 refer to similar types of number slots in the template. The other feature is whether two quantities appear in the same sentence.…”

Section: Global Featuresmentioning

confidence: 99%

The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers

Zhang

Wang

Zhang

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Solving mathematical word problems (MWPs) automatically is challenging, primarily due to the semantic gap between human-readable words and machine-understandable logics. Despite the long history dated back to the 1960s, MWPs have regained intensive attention in the past few years with the advancement of Artificial Intelligence (AI). Solving MWPs successfully is considered as a milestone towards general AI. Many systems have claimed promising results in self-crafted and small-scale datasets. However, when applied on large and diverse datasets, none of the proposed methods in the literature achieves high precision, revealing that current MWP solvers still have much room for improvement. This motivated us to present a comprehensive survey to deliver a clear and complete picture of automatic math problem solvers. In this survey, we emphasize on algebraic word problems, summarize their extracted features and proposed techniques to bridge the semantic gap, and compare their performance in the publicly accessible datasets. We also cover automatic solvers for other types of math problems such as geometric problems that require the understanding of diagrams. Finally, we identify several emerging research directions for the readers with interests in MWPs.

show abstract

“…The pruner discards such incompatible pairs. Heuristics to identify unit-compatible NUMSET pairs include three cases-1) direct string match, 2) synonymy/hypernymy relations from WordNet, 3) one unit is a nationality/ job 20 and the other unit is synonymous with person (Roy, 2017). 19 roughly, approximately, about, nearly, roundabout, around, circa, almost, approaching, pushing, more or less, in the neighborhood of, in the region of, on the order of,something like, give or take (a few), near to, close to, in the ballpark of 20 Lists of jobs, nationalities scraped from Wikipedia.…”

Section: Quantity Prunermentioning

confidence: 99%

EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference

Ravichander

Naik

Rosé

et al. 2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

Quantitative reasoning is a higher-order reasoning skill that any intelligent natural language understanding system can reasonably be expected to handle. We present EQUATE 1 (Evaluating Quantitative Understanding Aptitude in Textual Entailment), a new framework for quantitative reasoning in textual entailment. We benchmark the performance of 9 published NLI models on EQUATE, and find that on average, state-of-the-art methods do not achieve an absolute improvement over a majority-class baseline, suggesting that they do not implicitly learn to reason with quantities. We establish a new baseline Q-REAS that manipulates quantities symbolically. In comparison to the best performing NLI model, it achieves success on numerical reasoning tests (+24.2%), but has limited verbal reasoning capabilities (-8.1%). We hope our evaluation framework will support the development of models of quantitative reasoning in language understanding.

show abstract

Reasoning about Quantities in Natural Language

Cited by 109 publications

References 17 publications

Automatic generation of system test cases from use case specifications

Automatic generation of system test cases from use case specifications

The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers

EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference

Contact Info

Product

Resources

About