Coarse-to-fine Knowledge Graph Domain Adaptation based on Distantly-supervised Iterative Training

Abstract

Modern supervised learning neural network models require a large amount of manually labeled data, which makes the construction of domain-specific knowledge graphs timeconsuming and labor-intensive. In parallel, although there has been much research on named entity recognition and relation extraction based on distantly supervised learning, constructing a domain-specific knowledge graph from large collections of textual data without manual annotations is still an urgent problem to be solved. In response, we propose an integrated framework for adapting and re-learning knowledge graphs from one coarse domain (biomedical) to a finer-define domain (oncology). In this framework, we apply distant-supervision on cross-domain knowledge graph adaptation. Consequently, no manual data annotation is required to train the model. We introduce a novel iterative training strategy to facilitate the discovery of domainspecific named entities and triples. Experimental results indicate that the proposed framework can perform domain adaptation and construction of knowledge graph efficiently.

Surviving ChatGPT in healthcare

“…Datasets used to train language models can be substantial (1,29), often reaching hundreds of gigabytes, and they draw from various sources and domains (30)(31)(32)(33)(34). Consequently, even when trained on public data, these datasets can contain sensitive information, such as personally identifiable information (PII) including names, phone numbers, and addresses.…”

School Mathematics Textbooks in China

“…Transformer [1] was first proposed for translation tasks in NLP, which combines Multi-head Self Attention (MSA) with Feed-forward Networks (FFN) to offer a global perceptual field and multi-channel feature extraction capabilities. The subsequent development of the Transformer-based BERT [67] proved to be seminal in NLP, exhibiting exceptional performance across multiple language-related tasks [15,25,68]. Leveraging the great flexibility and scalability of the Transformer, researchers have started to train larger Transformer models, including GPT-1 [4], GPT-2 [5], GPT-3 [20], GPT-4 [22], T5 [3], PaLM [69], LLaMA [70] and others.…”