What is this article about? Generative summarization with the BERT model in the geosciences domain

Ma, Kai; Miao, Tian; Tan, Yongjian; Xie, Xuejing; Qiu, Qinjun

doi:10.1007/s12145-021-00695-2

Cited by 24 publications

(10 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The BERT model was also used by the authors of [ 7 , 8 , 16 , 36 , 46 ] for contextual representation in summarization tasks. The authors in [ 53 ] use the BERT model to automatically generate titles from a huge set of published literature or related work. Additionally, extractive summarization tasks using graph structures have been carried out by exploiting linguistic and statistical information included in sentences [ 9 ].…”

Section: Literature Reviewmentioning

confidence: 99%

N-GPETS: Neural Attention Graph-Based Pretrained Statistical Model for Extractive Text Summarization

Umair

Alam

Khan

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The extractive summarization approach involves selecting the source document’s salient sentences to build a summary. One of the most important aspects of extractive summarization is learning and modelling cross-sentence associations. Inspired by the popularity of Transformer-based Bidirectional Encoder Representations (BERT) pretrained linguistic model and graph attention network (GAT) having a sophisticated network that captures intersentence associations, this research work proposes a novel neural model N-GPETS by combining heterogeneous graph attention network with BERT model along with statistical approach using TF-IDF values for extractive summarization task. Apart from sentence nodes, N-GPETS also works with different semantic word nodes of varying granularity levels that serve as a link between sentences, improving intersentence interaction. Furthermore, proposed N-GPETS becomes more improved and feature-rich by integrating graph layer with BERT encoder at graph initialization step rather than employing other neural network encoders such as CNN or LSTM. To the best of our knowledge, this work is the first attempt to combine the BERT encoder and TF-IDF values of the entire document with a heterogeneous attention graph structure for the extractive summarization task. The empirical outcomes on benchmark news data sets CNN/DM show that the proposed model N-GPETS gets favorable results in comparison with other heterogeneous graph structures employing the BERT model and graph structures without the BERT model.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

N-GPETS: Neural Attention Graph-Based Pretrained Statistical Model for Extractive Text Summarization

Umair

Alam

Khan

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Li et al (2021) constructed a Chinese word segmentation algorithm based on a geological domain ontology assisted by a self-loop approach to better segment geological domain texts. Ma et al (2021) employed a deep learning model to train journal abstracts and titles in the field of Chinese geology. The resulting model was trained to perform automatic abstract construction.…”

Section: Text Mining In Geosciencementioning

confidence: 99%

“…Ma et al. (2021) employed a deep learning model to train journal abstracts and titles in the field of Chinese geology. The resulting model was trained to perform automatic abstract construction.…”

Section: Related Workmentioning

confidence: 99%

“…Named entity recognition (NER) has received much attention from the academic field and industry for many years (Nieh et al., 2021; Qiu, Xie, Wu, Tao, & Li, 2019; Qiu, Xie, Wu, & Tao, 2019; Zhou et al., 2021). The basis of geological domain knowledge graph construction is named entity recognition, that is, the recognition of specific categories of proper noun entities from unprocessed geological domain texts, and its accuracy directly affects the results of multiple natural language processing techniques in the geological domain (Ma et al., 2020, 2021; Wang et al., 2018).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Chinese Named Entity Recognition in the Geoscience Domain Based on BERT

Xie

Xu³

et al. 2022

Earth and Space Science

Self Cite

View full text Add to dashboard Cite

Geological reports are frequently used by geologists involved in geological surveys and scientific research to record the results and outcomes of geological surveys. With such a rich data source, a substantial amount of knowledge has yet to be mined and analyzed. This paper focuses on automatically information extraction from geological reports, namely, geological named entity recognition. Geological named entity recognition has an important role in data mining, knowledge discovery and Knowledge graph construction. Existing general named entity recognition models/tools are limited in the domain of geoscience due to the various language irregularities associated with geological text, such as informal sentence structures, several domain‐geoscience words, large character lengths and multiple combinations of independent words. We present Bidirectional encoder representations from transformers (BERT)‐(Bidirectional gated recurrent unit network) BiGRU‐ (Conditional random field) CRF, which is a deep learning‐based geological named entity recognition model that is designed specifically with these linguistic irregularities in mind. Based on the pretrained language model, an integrated deep learning model incorporating BERT, BiGRU and CRF is constructed to obtain character vectors rich in semantic information through the BERT pretrained language model to alleviate for the lack of specificity of static word vectors (e.g., word2vec) and to improve the extraction capability of complex geological entities. We demonstrate our proposed model by applying it to four test datasets, including a geoscience NER data set from regional geological reports, and by comparing its performance with those of five baseline models.

show abstract

“…In the field of geoscience, many types of geological data collections have accumulated over a long period of time due to the diversity of technical methods and research directions. In terms of data composition structure, massive geological data repositories include a large amount of structured data and unstructured data, especially textual data and geological map data (Ma et al., 2021; Qiu, Xie, & Wu, 2018; Qiu, Xie, Wu, & Li, 2018; Wu et al., 2017). Currently, a large number of geological reports are accumulated during geological investigations; each report contains information on a particular geological area, such as rocks, minerals, or hydrology, and the contents of these reports are typically saved in a variety of forms, including doc, pdf, jpg, tiff, and spatial data files (Qiu, Xie, Wu, & Li, 2019; Qiu, Xie, Wu, Tao, & Li, 2019; Wang et al., 2021, 2022).…”

Section: Introductionmentioning

confidence: 99%

GeoBERTSegmenter: Word Segmentation of Chinese Texts in the Geoscience Domain Using the Improved BERT Model

Wei

Liu²,

Xu³

et al. 2022

Earth and Space Science

Self Cite

View full text Add to dashboard Cite

show abstract

What is this article about? Generative summarization with the BERT model in the geosciences domain

Cited by 24 publications

References 33 publications

N-GPETS: Neural Attention Graph-Based Pretrained Statistical Model for Extractive Text Summarization

N-GPETS: Neural Attention Graph-Based Pretrained Statistical Model for Extractive Text Summarization

Chinese Named Entity Recognition in the Geoscience Domain Based on BERT

GeoBERTSegmenter: Word Segmentation of Chinese Texts in the Geoscience Domain Using the Improved BERT Model

Contact Info

Product

Resources

About