A machine‐learning approach to negation and speculation detection in clinical texts

Díaz, Noa P. Cruz; López, Manuel J. Maña; Vázquez, Jacinto Mata; Pachón, Victoria

doi:10.1002/asi.22679

Cited by 47 publications

(20 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the second stage a multi-class SVM was used to predict the final assertion prediction for each token [30]. Similar 2-step approach was applied to BioScope corpus by Diaz et al where each token in a sentence was classified as negation/speculation signal and a second classifier was used at a sentence level to determine the negation status of concept [31]. Goldin and Champan compared Naïve Bayes and decision trees with default NegEx rule on 207 sentences of clinical records with negation “not”.…”

Section: Related Workmentioning

confidence: 99%

DEEPEN: A negation detection system for clinical text incorporating dependency relation into NegEx

Mehrabi

Krishnan

Sohn

et al. 2015

Journal of Biomedical Informatics

109

View full text Add to dashboard Cite

In Electronic Health Records (EHRs), much of valuable information regarding patients’ conditions is embedded in free text format. Natural language processing (NLP) techniques have been developed to extract clinical information from free text. One challenge faced in clinical NLP is that the meaning of clinical entities is heavily affected by modifiers such as negation. A negation detection algorithm, NegEx, applies a simplistic approach that has been shown to be powerful in clinical NLP. However, due to the failure to consider the contextual relationship between words within a sentence, NegEx fails to correctly capture the negation status of concepts in complex sentences. Incorrect negation assignment could cause inaccurate diagnosis of patients’ condition or contaminated study cohorts. We developed a negation algorithm called DEEPEN to decrease NegEx’s false positives by taking into account the dependency relationship between negation words and concepts within a sentence using Stanford dependency parser. The system was developed and tested using EHR data from Indiana University (IU) and it was further evaluated on Mayo Clinic dataset to assess its generalizability. The evaluation results demonstrate DEEPEN, which incorporates dependency parsing into NegEx, can reduce the number of incorrect negation assignment for patients with positive findings, and therefore improve the identification of patients with the target clinical findings in EHRs.

show abstract

Section: Related Workmentioning

confidence: 99%

DEEPEN: A negation detection system for clinical text incorporating dependency relation into NegEx

Mehrabi

Krishnan

Sohn

et al. 2015

Journal of Biomedical Informatics

109

View full text Add to dashboard Cite

show abstract

“…They can be roughly categorized into manually crafted rule-based approaches [2,3,5,9] and supervised classification-based [1,6,7,16]. The former looks at lexico-syntactic patterns peculiar to negated expressions so as to spot them.…”

Section: Related Workmentioning

confidence: 99%

A hybrid approach to finding negated and uncertain expressions in biomedical documents

Fujikawa

Seki

Uehara

2012

Proceedings of the 2nd International Workshop on Managing Interoperability and compleXity in Health Systems

View full text Add to dashboard Cite

More and more biomedical documents are digitally written and stored. To make the most of the rich resources, it is crucial to precisely locate the information pertinent to users' interests. One of the obstacles in finding information in natural language text is negations, which deny or reverse the meaning of a sentence. This is especially problematic in the biomedical domain since scientific findings and clinical records often contain negated expressions to state negative effects or the absence of symptoms. Ignoring such negated expressions result in more irrelevant information and may even lead to false conclusions. Therefore, identifying negative words and their scopes are important sub-tasks in biomedical information processing. This paper reports on our ongoing work on a hybrid approach to negation identification combining statistical and heuristic approaches. Our approach is evaluated on three types of biomedical documents in comparison with an existing machine learning approach. In addition, the empirical results are manually analyzed to better understand the nature of the problems.

show abstract

“…Negation can be problematic with this approach (e.g., ‘not reliable’). A quick solution is to concatenate all of the occurrences of ‘not’ in the corpus to the following word before forming the DTM, though more sophisticated methods are also available …”

Section: Application Of Clustering and Tree Modelsmentioning

confidence: 99%

A practical guide to text mining with topic extraction

Karl¹,

Wisnowski²,

Rushing³

2015

WIREs Computational Stats

View full text Add to dashboard Cite

Text analytics continue to proliferate as mass volumes of unstructured but highly useful data are generated at unbounded rates. Vector space models for text data-in which documents are represented by rows and words by columns-provide a translation of this unstructured data into a format that may be analyzed with statistical and machine learning techniques. This approach gives excellent results in revealing common themes, clustering documents, clustering words, and in translating unstructured text fields (such as an open-ended survey response) to usable input variables for predictive modeling. After discussing the collection and processing of text, we explore properties and transformations of the document-term matrix (DTM). We show how the singular value decomposition may be used to drastically reduce the size of the document space while also setting the stage for automatic topic extraction, courtesy of the varimax rotation. This latent semantic analysis (LSA) approach produces factors that are compatible with graphical exploration and advanced analytics. We also explore Latent Dirichlet Allocation for topic analysis. We reference published R packages to implement the methods and conclude with a summary of other popular open-source and commercial software packages.

show abstract

A machine‐learning approach to negation and speculation detection in clinical texts

Cited by 47 publications

References 18 publications

DEEPEN: A negation detection system for clinical text incorporating dependency relation into NegEx

DEEPEN: A negation detection system for clinical text incorporating dependency relation into NegEx

A hybrid approach to finding negated and uncertain expressions in biomedical documents

A practical guide to text mining with topic extraction

Contact Info

Product

Resources

About