The Impact of Pretrained Language Models on Negation and Speculation Detection in Cross-Lingual Medical Text: Comparative Study

Abstract

Background Negation and speculation are critical elements in natural language processing (NLP)-related tasks, such as information extraction, as these phenomena change the truth value of a proposition. In the clinical narrative that is informal, these linguistic facts are used extensively with the objective of indicating hypotheses, impressions, or negative findings. Previous state-of-the-art approaches addressed negation and speculation detection tasks using rule-based methods, but in the last few years, models based on machine learning and deep learning exploiting morphological, syntactic, and semantic features represented as spare and dense vectors have emerged. However, although such methods of named entity recognition (NER) employ a broad set of features, they are limited to existing pretrained models for a specific domain or language. Objective As a fundamental subsystem of any information extraction pipeline, a system for cross-lingual and domain-independent negation and speculation detection was introduced with special focus on the biomedical scientific literature and clinical narrative. In this work, detection of negation and speculation was considered as a sequence-labeling task where cues and the scopes of both phenomena are recognized as a sequence of nested labels recognized in a single step. Methods We proposed the following two approaches for negation and speculation detection: (1) bidirectional long short-term memory (Bi-LSTM) and conditional random field using character, word, and sense embeddings to deal with the extraction of semantic, syntactic, and contextual patterns and (2) bidirectional encoder representations for transformers (BERT) with fine tuning for NER. Results The approach was evaluated for English and Spanish languages on biomedical and review text, particularly with the BioScope corpus, IULA corpus, and SFU Spanish Review corpus, with F-measures of 86.6%, 85.0%, and 88.1%, respectively, for NeuroNER and 86.4%, 80.8%, and 91.7%, respectively, for BERT. Conclusions These results show that these architectures perform considerably better than the previous rule-based and conventional machine learning–based systems. Moreover, our analysis results show that pretrained word embedding and particularly contextualized embedding for biomedical corpora help to understand complexities inherent to biomedical text.

Negation recognition in clinical natural language processing using a combination of the NegEx algorithm and a convolutional neural network

“…In addition, deep learning approaches have been implemented to further improve negation recognition [ 17 , 38 , 39 ]. For instance, context-independent and context-dependent pretrained transformers models achieved an F-score performance of over 85% for negation recognition in medical text outperforming rule-based methods [ 40 ]. The authors analyzed the most frequent false negatives and false positives for negation and speculation recognition and concluded that the ambiguity of some grammatical structures led their model to misclassify some tokens resulting in a decreased performance [ 40 ].…”

Coding linguistic elements in clinical interactions: a step-by-step guide for analyzing communication form

“…Reliability of automated techniques is lower for more complex linguistic elements that require interpretation, such as coding utterance valence when negations are used (e.g. [ 51 , 52 ]). Manual coding in addition to automated text processing is therefore necessary to guarantee consistent coding [ 53 ].…”

Analyzing transfer learning impact in biomedical cross-lingual named entity recognition and normalization

“…BMEWO-V is similar to other previous encodings [ 35 ]; however, we introduce the V tag to allow the representation of overlapping or nested entities which are usual phenomena in these types of texts. Additionally, we tested the BMEWO-V enconding format in previous works [ 16 , 36 ]. Finally, the BRAT format is transformed into sentences annotated in the CoNLL-2003 format [ 37 ].…”