Using pretraining and text mining methods to automatically extract the chemical scientific data

Abstract

Purpose In computational chemistry, the chemical bond energy (pKa) is essential, but most pKa-related data are submerged in scientific papers, with only a few data that have been extracted by domain experts manually. The loss of scientific data does not contribute to in-depth and innovative scientific data analysis. To address this problem, this study aims to utilize natural language processing methods to extract pKa-related scientific data in chemical papers. Design/methodology/approach Based on the previous Bert-CRF model combined with dictionaries and rules to resolve the problem of a large number of unknown words of professional vocabulary, in this paper, the authors proposed an end-to-end Bert-CRF model with inputting constructed domain wordpiece tokens using text mining methods. The authors use standard high-frequency string extraction techniques to construct domain wordpiece tokens for specific domains. And in the subsequent deep learning work, domain features are added to the input. Findings The experiments show that the end-to-end Bert-CRF model could have a relatively good result and can be easily transferred to other domains because it reduces the requirements for experts by using automatic high-frequency wordpiece tokens extraction techniques to construct the domain wordpiece tokenization rules and then input domain features to the Bert model. Originality/value By decomposing lots of unknown words with domain feature-based wordpiece tokens, the authors manage to resolve the problem of a large amount of professional vocabulary and achieve a relatively ideal extraction result compared to the baseline model. The end-to-end model explores low-cost migration for entity and relation extraction in professional fields, reducing the requirements for experts.

Assessing the Efficacy of Air Pollution Mitigation in Beijing: Insights from GIS and Machine Learning Analyses

“…Employing sentiment analysis proves instrumental in discerning the nuanced perspectives of netizens regarding fluctuations in air quality across temporal intervals . Predominantly, such sentiment analysis methodologies leverage sophisticated computational models, notably Large Language Models (Wu, Xiang, et al, 2024), GenAI models (Xiang et al, 2024, BERT models (Pang et al, 2019), andMultimodal Transformer (Lyu, Dong, et al, 2022b).…”

Enhancing Attention Models via Multi-head Collaboration

“…With the development of deep learning models (Lyu et al, 2022aPang et al, 2019;Dong et al, 2023;Lin et al, 2023;Feng et al, 2022;Bu et al, 2021;Zhou et al, 2024a;Zhuang and Al Hasan, 2022;Li et al, 2024Zhou et al, 2023Zhou et al, , 2024bZhang et al, 2024a,b;Mo et al, 2022),…”

Extracting structured seed-mediated gold nanorod growth procedures from scientific text with LLMs

“…Natural language processing (NLP) has been successfully applied in the chemical, medical, and materials sciences to produce structured data from unstructured text using methods and models such as pattern recognition, recurrent neural networks, and language models. 28,28–52…”