BioWordVec, improving biomedical word embeddings with subword information and MeSH

Zhang, Yijia; Chen, Qingyu; Yang, Zhihao; Lin, Hongfei; Lu, Zhiyong

doi:10.1038/s41597-019-0055-0

Cited by 369 publications

(263 citation statements)

References 29 publications

Supporting

Mentioning

242

Contrasting

Order By: Relevance

“…One established trend is a form of word embeddings that represent the semantic, using high dimensional vectors (Chiu et al, 2016;Wang et al, 2018c;Zhang et al, 2019). Similar methods also have been derived to improve embeddings of word sequences by introducing sentence embeddings .…”

Section: Related Workmentioning

confidence: 99%

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Peng

Yan

2019

Proceedings of the 18th BioNLP Workshop and Shared Task

Self Cite

614

462

View full text Add to dashboard Cite

Inspired by the success of the General Language Understanding Evaluation benchmark, we introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to facilitate research in the development of pre-training language representations in the biomedicine domain. The benchmark consists of five tasks with ten datasets that cover both biomedical and clinical texts with different dataset sizes and difficulties. We also evaluate several baselines based on BERT and ELMo and find that the BERT model pre-trained on PubMed abstracts and MIMIC-III clinical notes achieves the best results. We make the datasets, pre-trained models, and codes publicly available at https://github.com/ ncbi-nlp/BLUE_Benchmark.

show abstract

Section: Related Workmentioning

confidence: 99%

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Peng

Yan

2019

Proceedings of the 18th BioNLP Workshop and Shared Task

Self Cite

614

462

View full text Add to dashboard Cite

show abstract

“…For this setting, we found approximately 1,000 unique publications, screened them for relevance, and, finally, included roughly 100 into this survey. [16] BioWordVec [17] BioSentVec [18] Flair('pubmed-X')…”

Section: Design and Goals Of This Surveymentioning

confidence: 99%

Medical Information Extraction in the Age of Deep Learning

Hahn

Oleynik

2020

Yearb Med Inform

View full text Add to dashboard Cite

Objectives: We survey recent developments in medical Information Extraction (IE) as reported in the literature from the past three years. Our focus is on the fundamental methodological paradigm shift from standard Machine Learning (ML) techniques to Deep Neural Networks (DNNs). We describe applications of this new paradigm concentrating on two basic IE tasks, named entity recognition and relation extraction, for two selected semantic classes—diseases and drugs (or medications)—and relations between them. Methods: For the time period from 2017 to early 2020, we searched for relevant publications from three major scientific communities: medicine and medical informatics, natural language processing, as well as neural networks and artificial intelligence. Results: In the past decade, the field of Natural Language Processing (NLP) has undergone a profound methodological shift from symbolic to distributed representations based on the paradigm of Deep Learning (DL). Meanwhile, this trend is, although with some delay, also reflected in the medical NLP community. In the reporting period, overwhelming experimental evidence has been gathered, as illustrated in this survey for medical IE, that DL-based approaches outperform non-DL ones by often large margins. Still, small-sized and access-limited corpora create intrinsic problems for data-greedy DL as do special linguistic phenomena of medical sublanguages that have to be overcome by adaptive learning strategies. Conclusions: The paradigm shift from (feature-engineered) ML to DNNs changes the fundamental methodological rules of the game for medical NLP. This change is by no means restricted to medical IE but should also deeply influence other areas of medical informatics, either NLP- or non-NLP-based.

show abstract

“…In the results, we compare two variants of DeepRank: BioDeepRank refers to the model with the modified aggregation network and weighting mechanism, and using word embeddings for the biomedical domain [15]; Attn-BioDeepRank refers to the final model that additionally replaces the recurrent layer by a self-attention layer. 2 Neural Ranking Models.…”

Section: Methodsmentioning

confidence: 99%

Calling Attention to Passages for Biomedical Question Answering

Almeida

Matos

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Question answering can be described as retrieving relevant information for questions expressed in natural language, possibly also generating a natural language answer. This paper presents a pipeline for document and passage retrieval for biomedical question answering built around a new variant of the DeepRank network model in which the recursive layer is replaced by a self-attention layer combined with a weighting mechanism. This adaptation halves the total number of parameters and makes the network more suited for identifying the relevant passages in each document. The overall retrieval system was evaluated on the BioASQ tasks 6 and 7, achieving similar retrieval performance when compared to more complex network architectures.

show abstract

BioWordVec, improving biomedical word embeddings with subword information and MeSH

Cited by 369 publications

References 29 publications

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Medical Information Extraction in the Age of Deep Learning

Calling Attention to Passages for Biomedical Question Answering

Contact Info

Product

Resources

About