Learning Numeracy: A Simple Yet Effective Number Embedding Approach Using Knowledge Graph

Duan, Hanyu; Yang, Yindong; Tam, Kar Yan

doi:10.18653/v1/2021.findings-emnlp.221

Cited by 8 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed NR model is compared with previous SOTA phenotyping methods. In the unsupervised setting, we consider unsupervised baseline methods including NCBO, 6 NCR, 8 and the unsupervised model by Zhang et al 12 In the supervised setting, we consider ClinicalBERT 27 (which is fine-tuned in separate for phenotyping) and the supervised model by Zhang et al 12 The NCBO, NCR, and fine-tuned ClinicalBERT are selected as they show overall better performance than other baseline phenotyping methods in corresponding settings, as demonstrated by Zhang et al 12 We decide not to use recent NR models, [15][16][17][18] as baseline methods because none of them considers clinical knowledge and it is costly to adapt them to the clinical domain.…”

Section: Baselines and Evaluation Methodsmentioning

confidence: 99%

“…We decide not to use recent NR models, 15 –18 as baseline methods because none of them considers clinical knowledge and it is costly to adapt them to the clinical domain.…”

Section: Methodsmentioning

confidence: 99%

“…However, none of these methods mentioned above can reason with numbers in clinical text, for example, "temperature 102°F" suggesting Fever. Recent works in NR publish new datasets 14 and develop NR capabilities with deep learning [15][16][17][18] in the respective domains rather than the clinical domain. For instance, Geva et al 19 show better performance on tasks involving numeracy, such as math word problems, and reading comprehension by using artificially created data.…”

Section: Impact Statementmentioning

confidence: 99%

See 2 more Smart Citations

Phenotyping in clinical text with unsupervised numerical reasoning for patient stratification

Tanwar¹,

Zhang²,

Ive³

et al. 2022

Exp Biol Med (Maywood)

View full text Add to dashboard Cite

Phenotypic information of patients, as expressed in clinical text, is important in many clinical applications such as identifying patients at risk of hard-to-diagnose conditions. Extracting and inferring some phenotypes from clinical text requires numerical reasoning, for example, a temperature of 102°F suggests the phenotype Fever. However, while current state-of-the-art phenotyping models using natural language processing (NLP) are in general very efficient in extracting phenotypes, they struggle to extract phenotypes that require numerical reasoning. In this article, we propose a novel unsupervised method that leverages external clinical knowledge and contextualized word embeddings by ClinicalBERT for numerical reasoning in different phenotypic contexts. Experiments show that the proposed method achieves significant improvement against unsupervised baseline methods with absolute increase in generalized Recall and F1 scores of up to 79% and 71%, respectively. Also, the proposed method outperforms supervised baseline methods with absolute increase in generalized Recall and F1 scores of up to 70% and 44%, respectively. In addition, we validate the methodology on clinical use cases where the detected phenotypes significantly contribute to patient stratification systems for a set of diseases, namely, HIV and myocardial infarction (heart attack). Moreover, we find that these phenotypes from clinical text can be used to impute the missing values in structured data, which enrich and improve data quality.

show abstract

Section: Baselines and Evaluation Methodsmentioning

confidence: 99%

“…We decide not to use recent NR models, 15 –18 as baseline methods because none of them considers clinical knowledge and it is costly to adapt them to the clinical domain.…”

Section: Methodsmentioning

confidence: 99%

Section: Impact Statementmentioning

confidence: 99%

See 1 more Smart Citation

Phenotyping in clinical text with unsupervised numerical reasoning for patient stratification

Tanwar¹,

Zhang²,

Ive³

et al. 2022

Exp Biol Med (Maywood)

View full text Add to dashboard Cite

show abstract

“…Please note the work by [38] publishes one unsupervised and one supervised model hence we compare the proposed NR model with both. We decide not to compare with recent numerical reasoning models (such as [34,9,31,13]) as none of them incorporates clinical knowledge and we find it costly to adapt them to the clinical domain.…”

Section: Baselines and Evaluation Methodsmentioning

confidence: 99%

“…Numerical Reasoning: Recent works publish new datasets for numerical reasoning [39] and utilise deep learning based models to develop the numerical reasoning skills [34,9,31,13] in respective domains other than the clinical domain. For example, [11] shows gains by using artificially created data on various tasks involving numeracy such as math word problems and reading comprehension.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledge

Tanwar¹,

Zhang²,

Ive³

et al. 2022

Preprint

View full text Add to dashboard Cite

Extracting phenotypes from clinical text has been shown to be useful for a variety of clinical use cases such as identifying patients with rare diseases. However, reasoning with numerical values remains challenging for phenotyping in clinical text, for example, temperature 102F representing Fever. Current state-of-the-art phenotyping models are able to detect general phenotypes, but perform poorly when they detect phenotypes requiring numerical reasoning. We present a novel unsupervised methodology leveraging external knowledge and contextualized word embeddings from ClinicalBERT for numerical reasoning in a variety of phenotypic contexts. Comparing against unsupervised benchmarks, it shows a substantial performance improvement with absolute gains on generalized Recall and F1 scores up to 79% and 71%, respectively. In the supervised setting, it also surpasses the performance of alternative approaches with absolute gains on generalized Recall and F1 scores up to 70% and 44%, respectively.

show abstract

Does a language model “understand” high school math? A survey of deep learning based word problem solvers

Sundaram,

Gurajada,

Padmanabhan

et al. 2024

WIREs Data Min & Knowl

View full text Add to dashboard Cite

From the latter half of the last decade, there has been a growing interest in developing algorithms for automatically solving mathematical word problems (MWP). It is a challenging and unique task that demands blending surface level text pattern recognition with mathematical reasoning. In spite of extensive research, we still have a lot to explore for building robust representations of elementary math word problems and effective solutions for the general task. In this paper, we critically examine the various models that have been developed for solving word problems, their pros and cons and the challenges ahead. In the last 2 years, a lot of deep learning models have recorded competing results on benchmark datasets, making a critical and conceptual analysis of literature highly useful at this juncture. We take a step back and analyze why, in spite of this abundance in scholarly interest, the predominantly used experiment and dataset designs continue to be a stumbling block. From the vantage point of having analyzed the literature closely, we also endeavor to provide a road‐map for future math word problem research.This article is categorized under: Technologies > Machine Learning Technologies > Artificial Intelligence Fundamental Concepts of Data and Knowledge > Knowledge Representation

show abstract

Learning Numeracy: A Simple Yet Effective Number Embedding Approach Using Knowledge Graph

Cited by 8 publications

References 13 publications

Phenotyping in clinical text with unsupervised numerical reasoning for patient stratification

Phenotyping in clinical text with unsupervised numerical reasoning for patient stratification

Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledge

Does a language model “understand” high school math? A survey of deep learning based word problem solvers

Contact Info

Product

Resources

About