Evaluating sentence representations for biomedical text: Methods and experimental results

Tawfik, Noha S.; Spruit, Marco

doi:10.1016/j.jbi.2020.103396

Cited by 32 publications

(20 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As seen in Table 9, we could not single out a particular contextualized word embedding to utilize, as the use of word embedding may vary according to the various reasons: type of task (OIE, RE, or sentiment analysis), dataset domain (news, bio-medical data, or financial data), and the computational power available to the user. This is also in agreement with other papers that extensively compared embeddings in various tasks and found that the most suitable one is highly dependent on the task and data nature [51,52]. Table 9.…”

Section: Relation Extraction Results Discussionsupporting

confidence: 91%

Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

Sarhan

Spruit

2020

Applied Sciences

Self Cite

View full text Add to dashboard Cite

Various tasks in natural language processing (NLP) suffer from lack of labelled training data, which deep neural networks are hungry for. In this paper, we relied upon features learned to generate relation triples from the open information extraction (OIE) task. First, we studied how transferable these features are from one OIE domain to another, such as from a news domain to a bio-medical domain. Second, we analyzed their transferability to a semantically related NLP task, namely, relation extraction (RE). We thereby contribute to answering the question: can OIE help us achieve adequate NLP performance without labelled data? Our results showed comparable performance when using inductive transfer learning in both experiments by relying on a very small amount of the target data, wherein promising results were achieved. When transferring to the OIE bio-medical domain, we achieved an F-measure of 78.0%, only 1% lower when compared to traditional learning. Additionally, transferring to RE using an inductive approach scored an F-measure of 67.2%, which was 3.8% lower than training and testing on the same task. Hereby, our analysis shows that OIE can act as a reliable source task.

show abstract

Section: Relation Extraction Results Discussionsupporting

confidence: 91%

Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

Sarhan

Spruit

2020

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, the Pearson correlation score achieved by the RoBERTa-mimic was 0.8705; however, the RoBERTa-base yielded a higher performance of 0.8778. Tawfik et al [45] have similarly observed that the PubMed pretrained BioBERT did not outperform the corresponding general BERT model pretrained using English text on clinical STS. In the clinical STS task, using STS-General (an STS corpus annotated in the general English domain) as an extra training set in addition to STS-Clinic could efficiently improve performances for transformer-based models.…”

Section: Experiments Findingsmentioning

confidence: 99%

Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models

Yang¹,

He²,

Zhang³

et al. 2020

JMIR Med Inform

View full text Add to dashboard Cite

Background Semantic textual similarity (STS) is one of the fundamental tasks in natural language processing (NLP). Many shared tasks and corpora for STS have been organized and curated in the general English domain; however, such resources are limited in the biomedical domain. In 2019, the National NLP Clinical Challenges (n2c2) challenge developed a comprehensive clinical STS dataset and organized a community effort to solicit state-of-the-art solutions for clinical STS. Objective This study presents our transformer-based clinical STS models developed during this challenge as well as new models we explored after the challenge. This project is part of the 2019 n2c2/Open Health NLP shared task on clinical STS. Methods In this study, we explored 3 transformer-based models for clinical STS: Bidirectional Encoder Representations from Transformers (BERT), XLNet, and Robustly optimized BERT approach (RoBERTa). We examined transformer models pretrained using both general English text and clinical text. We also explored using a general English STS dataset as a supplementary corpus in addition to the clinical training set developed in this challenge. Furthermore, we investigated various ensemble methods to combine different transformer models. Results Our best submission based on the XLNet model achieved the third-best performance (Pearson correlation of 0.8864) in this challenge. After the challenge, we further explored other transformer models and improved the performance to 0.9065 using a RoBERTa model, which outperformed the best-performing system developed in this challenge (Pearson correlation of 0.9010). Conclusions This study demonstrated the efficiency of utilizing transformer-based models to measure semantic similarity for clinical text. Our models can be applied to clinical applications such as clinical text deduplication and summarization.

show abstract

“…Sentence embeddings have been evaluated with several tasks regarding NLP in the medical domain. For example, the authors of [36] performed a comprehensive evaluation of different sentence embedding based models for different tasks, such as semantic similarity, question answering or text-classification. Although some of the models evaluated showed promising results, there was no clear winner that beat the other models for all the tasks.…”

Section: Feature Engineeringmentioning

confidence: 99%

Ontology-driven aspect-based sentiment analysis classification: An infodemiological case study regarding infectious diseases in Latin America

García-Díaz

Cánovas-García

Valencia‐García

2020

Future Generation Computer Systems

View full text Add to dashboard Cite

Infodemiology is the process of mining unstructured and textual data so as to provide public health officials and policymakers with valuable information regarding public health. The appearance of this new data source, which was previously unimaginable, has opened up a new way in which to improve public health systems, resulting in better communication policies and better detection systems. However, the unstructured nature of the Internet, along with the complexity of the infectious disease domain, prevents the information extracted from being easily understood. Moreover, when dealing with languages other than English, for which some of the most common Natural Language Processing resources are not available, the correct exploitation of this data becomes even more difficult. We intend to fill these gaps proposing an ontology-driven aspect-based sentiment analysis with which to measure the general public's opinions as regards infectious diseases when expressed in Spanish by employing a case study of tweets concerning the Zika, Dengue and Chikungunya viruses in Latin America. Our proposal is based on two technologies. We first use ontologies in order to model the infectious disease domain with concepts such as risks, symptoms, transmission methods or drugs, among other concepts. We then measure the relationship between these concepts in order to determine the degree to which one concept influences other concepts. This new information is subsequently applied in order to build an aspect-based sentiment analysis model based on statistical and linguistic features. This is done by applying deep-learning models. Our proposal is available on a web platform, where users can see the sentiment for each concept at a glance and analyse how each concept influences the sentiment of the others.

show abstract

Evaluating sentence representations for biomedical text: Methods and experimental results

Cited by 32 publications

References 42 publications

Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models

Ontology-driven aspect-based sentiment analysis classification: An infodemiological case study regarding infectious diseases in Latin America

Contact Info

Product

Resources

About