Transfer Learning for Named Entity Recognition in Financial and Biomedical Documents

Francis, Sumam; Landeghem, Jordy Van; Moens, Marie‐Francine

doi:10.3390/info10080248

Cited by 33 publications

(20 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CNN YOLO is also used for object recognition and feature extraction in complex documents such as comics [111]. • BERT: Few studies [83], [13] reported the use of BERT for extracting features from the data. It is suitable for various types of language processing related task in NLP.…”

Section: ) Named Entity Recognition (Ner)mentioning

confidence: 99%

“…Though BERT is a powerful NLP model, using it for NER without fine-tuning it on the NER dataset will not give better results. Fine-tuning BERT for NER performance improvement in financial and biomedical documents utilizing the combination of BERT and word embedding is discussed in the study [83]. Utilizing and fine-tuning the BERT model for achieving document-level relation extraction using DocRED-a large scale open-domain document-level relation extraction dataset shows improvement in F1 measure in [13].…”

Section: ) Named Entity Recognition (Ner)mentioning

confidence: 99%

See 1 more Smart Citation

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

et al. 2021

View full text Add to dashboard Cite

The unstructured data impacts 95% of the organizations and costs them millions of dollars annually. If managed well, it can significantly improve business productivity. The traditional information extraction techniques are limited in their functionality, but AI-based techniques can provide a better solution. A thorough investigation of AI-based techniques for automatic information extraction from unstructured documents is missing in the literature. The purpose of this Systematic Literature Review (SLR) is to recognize, and analyze research on the techniques used for automatic information extraction from unstructured documents and to provide directions for future research. The SLR guidelines proposed by Kitchenham and Charters were adhered to conduct a literature search on various databases between 2010 and 2020. We found that: 1. The existing information extraction techniques are template-based or rule-based, 2. The existing methods lack the capability to tackle complex document layouts in real-time situations such as invoices and purchase orders, 3.The datasets available publicly are task-specific and of low quality. Hence, there is a need to develop a new dataset that reflects real-world problems. Our SLR discovered that AI-based approaches have a strong potential to extract useful information from unstructured documents automatically. However, they face certain challenges in processing multiple layouts of the unstructured documents. Our SLR brings out conceptualization of a framework for construction of high-quality unstructured documents dataset with strong data validation techniques for automated information extraction. Our SLR also reveals a need for a close association between the businesses and researchers to handle various challenges of the unstructured data analysis.

show abstract

Section: ) Named Entity Recognition (Ner)mentioning

confidence: 99%

Section: ) Named Entity Recognition (Ner)mentioning

confidence: 99%

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The benefits of TL have previously been investigated for the purposes of biomedical NER (Sun and Yang, 2019;Francis et al, 2019) and RE Peng et al, 2019;Hafiane et al, 2020). Recent work has been aimed at solving the challenges of imbalanced relation distribution, linguistic variation and partial transfer using relationgated adversarial learning , and capturing dependency tree information using TreeLSTM models (Legrand et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

SAFFRON: tranSfer leArning For Food-disease RelatiOn extractioN

Cenikj¹,

Eftimov²,

Seljak³

2021

Proceedings of the 20th Workshop on Biomedical Language Processing

View full text Add to dashboard Cite

The accelerating growth of big data in the biomedical domain, with an endless amount of electronic health records and more than 30 million citations and abstracts in PubMed, introduces the need for automatic structuring of textual biomedical data. In this paper, we develop a method for detecting relations between food and disease entities from raw text. Due to the lack of annotated data on food with respect to health, we explore the feasibility of transfer learning by training BERT-based models on existing datasets annotated for the presence of cause and treat relations among different types of biomedical entities, and using them to recognize the same relations between food and disease entities in a dataset created for the purposes of this study. The best models achieve macro averaged F1 scores of 0.847 and 0.900 for the cause and treat relations, respectively.

show abstract

“…It also has a rapid development in named entity recognition [28,29]. In the cross-language NER tasks [30] or cross-domain dataset NER tasks [31], the method based on transfer learning has been well proved to be usable.…”

Section: Related Workmentioning

confidence: 99%

POISE: Efficient Cross-Domain Chinese Named Entity Recognization via Transfer Learning

Sheng

Wumaier

2020

Symmetry

View full text Add to dashboard Cite

To improve the performance of deep learning methods in case of a lack of labeled data for entity annotation in entity recognition tasks, this study proposes transfer learning schemes that combine the character to be the word to convert low-resource data symmetry into high-resource data. We combine character embedding, word embedding, and the embedding of the label features using high- and low-resource data based on the BiLSTM-CRF model, and perform the feature-transfer and parameter-sharing tasks in two domains of the BiLSTM network to annotate with zero resources. Before transfer learning, we must first calculate the label similarity between two different domains and select the label features with large similarity for feature transfer mapping. All training parameters of the source domain in the model are shared during the BiLSTM network processing and CRF layer. In addition, we also use the method of combining characters and words to reduce the problem of word segmentation across domains and reduce the error rate in label mapping. The results of experiments show that in terms of the overall F1 score, the proposed model without supervision was superior by 9.76 percentage points to the general parametric shared transfer learning method, and by 9.08 and 12.38 percentage points, respectively, to two recent high–low resource learning methods. The proposed scheme improves performance in terms of transfer learning between the high- and low-resource data and can identify the predicted data in the target domain.

show abstract

Transfer Learning for Named Entity Recognition in Financial and Biomedical Documents

Cited by 33 publications

References 14 publications

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

Efficient Automated Processing of the Unstructured Documents Using Artificial Intelligence: A Systematic Literature Review and Future Directions

SAFFRON: tranSfer leArning For Food-disease RelatiOn extractioN

POISE: Efficient Cross-Domain Chinese Named Entity Recognization via Transfer Learning

Contact Info

Product

Resources

About