Construction of Machine-Labeled Data for Improving Named Entity Recognition by Transfer Learning

Abstract

Deep neural networks (DNNs) require a large amount of manually labeled training data to make significant achievements. However, manual labeling is laborious and costly. In this study, we propose a method for automatically generating training data and effectively using the generated data to reduce the labeling cost. The generated data (called ''machine-labeled data'') is generated using a bagging-based bootstrapping approach. However, using the machine-labeled data does not guarantee high performance because of errors in the automatic labeling. In order to reduce the impact of mislabeling, we applied a transfer learning approach. The effect of our proposed method was verified with two versions of DNN-based named entity recognition (NER) models: bidirectional LSTM-CRF and vanilla BERT. We conducted NER tasks in two languages (English and Korean). The proposed method results in average F1 scores of 78.87% (3.9% point improvement) with bidirectional LSTM-CRF and 82.08% (1% point improvement) with BERT on three Korean NER datasets. In English, the performance increased by an average of 0.45% points with the two DNN-based models. The proposed NER systems outperform the baseline systems in both languages without the need for additional manual labeling. INDEX TERMS Named entity recognition, bootstrapping, bagging, transfer learning, deep learning.

ProtagonistTagger -- a Tool for Entity Linkage of Persons in Texts from Various Languages and Domains

“…The protagonistTagger tool achieves high results on all the tested novels -precision and recall above 83% (see Table 2). The tool's performance on Test small names shows that it can be successfully used for new distinct novels 2 The adapted tool along with the new datasets: https://zenodo.org/record/ 5060232 to create a larger corpus of annotated texts. The best proof of novels' diversity in the test sets is the tool's performance, whose precision varies from 79% to even 96% for different novels.…”

“…The novel is a particular type of text in terms of writing style, the links between sentences, the plot's complexity, the number of characters, etc. NER phase of the linkage process in the literary domain uses a pretrained standard NER model fine-tuned with data from literary domain annotated with general tag person in a semi-automatic way [2]. It was necessary due to the relatively low performance of standard models trained primarily on web data such as blogs, news, and comments [1,4].…”

Transformer Based Language Identification for Malayalam-English Code-Mixed Text

“…Researchers have also exploited other areas of code-switching schemes, like information retrieval systems e.g. : Automatic Aspect Extraction [48], Polarity Identification [49], Sequence labelling tasks such as Named Entity Recognition [50], Automatic Speech Recognition [51], and Parsing [52]. From the related works we observed that many studies have been conducted in the language identification task in many language pairs but none of them are addressing Malayalam-English due to lack of publicly available data set.…”

Research on a Rolling Bearing Fault Detection Method With Wavelet Convolution Deep Transfer Learning

“…Transfer learning is a method that uses existing knowledge to solve problems in different but related fields. Transfer learning methods have successfully been applied in image recognition [5], speech recognition [6], text recognition [7], and other fields. Research on the combination of transfer learning and fault diagnosis has gradually become a research hotspot.…”