Chinese Named Entity Recognition in the Geoscience Domain Based on BERT

Abstract

Geological reports are frequently used by geologists involved in geological surveys and scientific research to record the results and outcomes of geological surveys. With such a rich data source, a substantial amount of knowledge has yet to be mined and analyzed. This paper focuses on automatically information extraction from geological reports, namely, geological named entity recognition. Geological named entity recognition has an important role in data mining, knowledge discovery and Knowledge graph construction. Existing general named entity recognition models/tools are limited in the domain of geoscience due to the various language irregularities associated with geological text, such as informal sentence structures, several domain‐geoscience words, large character lengths and multiple combinations of independent words. We present Bidirectional encoder representations from transformers (BERT)‐(Bidirectional gated recurrent unit network) BiGRU‐ (Conditional random field) CRF, which is a deep learning‐based geological named entity recognition model that is designed specifically with these linguistic irregularities in mind. Based on the pretrained language model, an integrated deep learning model incorporating BERT, BiGRU and CRF is constructed to obtain character vectors rich in semantic information through the BERT pretrained language model to alleviate for the lack of specificity of static word vectors (e.g., word2vec) and to improve the extraction capability of complex geological entities. We demonstrate our proposed model by applying it to four test datasets, including a geoscience NER data set from regional geological reports, and by comparing its performance with those of five baseline models.

Transformer-Based Named Entity Recognition in Construction Supply Chain Risk Management in Australia

“…Moreover, other scholarly endeavours have demonstrated automated techniques for extracting spatiotemporal and semantic information from geological documents. These techniques are crucial for tasks such as data mining, knowledge discovery, and constructing knowledge graphs [41,42]. The narrative above explains the importance and modern approaches used in automating the extraction of geological news information.…”

ENEX-FP: A BERT-Based Address Recognition Model

“…Paper [19] introduces the BERT-BiGRU-CRF model, which is specially designed for these linguistic irregularities. The accuracy of this model on the MSRA dataset is 0.981, higher than other recognition models such as BERT-BILSTM-CRF.…”

Geographic Named Entity Recognition by Employing Natural Language Processing and an Improved BERT Model

“…The BERT model's "masked language model" (MLM) can fuse the left and right contexts of the current word. BERT has achieved remarkable results in tasks such as named-entity recognition [28], text classification, machine translation [29], etc. The next sentence prediction (NSP) captures sentence-level representations and obtains semantically rich, high-quality feature representation vectors.…”