ChEMU 2020: Natural Language Processing Methods Are Effective for Information Extraction From Chemical Patents

Abstract

Chemical patents represent a valuable source of information about new chemical compounds, which is critical to the drug discovery process. Automated information extraction over chemical patents is, however, a challenging task due to the large volume of existing patents and the complex linguistic properties of chemical patents. The Cheminformatics Elsevier Melbourne University (ChEMU) evaluation lab 2020, part of the Conference and Labs of the Evaluation Forum 2020 (CLEF2020), was introduced to support the development of advanced text mining techniques for chemical patents. The ChEMU 2020 lab proposed two fundamental information extraction tasks focusing on chemical reaction processes described in chemical patents: (1) chemical named entity recognition, requiring identification of essential chemical entities and their roles in chemical reactions, as well as reaction conditions; and (2) event extraction, which aims at identification of event steps relating the entities involved in chemical reactions. The ChEMU 2020 lab received 37 team registrations and 46 runs. Overall, the performance of submissions for these tasks exceeded our expectations, with the top systems outperforming strong baselines. We further show the methods to be robust to variations in sampling of the test data. We provide a detailed overview of the ChEMU 2020 corpus and its annotation, showing that inter-annotator agreement is very strong. We also present the methods adopted by participants, provide a detailed analysis of their performance, and carefully consider the potential impact of data leakage on interpretation of the results. The ChEMU 2020 Lab has shown the viability of automated methods to support information extraction of key information in chemical patents.

Ensemble of deep masked language models for effective named entity recognition in multi-domain corpora

“…The ChEMU 2020 benchmark dataset 1 [He et al, 2021] contains snippets sampled from 170 English patents from the European Patent Office and the United States Patent and Trademark Office [He et al, 2020a[He et al, ,b, 2021. As shown in Fig.…”

Identifying Chemical Reactions and Their Associated Attributes in Patents

“…Here, we discuss the previous literature within this domain. He et al (2021) used a CRF-based model for NER and a rulebased system for EE. For NER, they developed the BANNER NER system (Leaman and Gonzalez, 2008) which uses lexical, syntactic, and contextual features in a CRF model.…”

“…Chemical patents are a significant source of information about novel chemicals and chemical reactions. New chemical compound discovery plays a vital role in the chemical and pharmaceutical industry, and chemical patents are the first venue this information is disclosed ( He et al, 2021 ). Unfortunately, there has been a rapid growth of chemical patents in recent years, and with the increasing volume, the manual cataloging of these chemicals and chemical reactions is become laborious and time-intensive, making it difficult for researchers to keep up with the current state of the art.…”

“…A chemical reaction typically includes an ordered sequence of reaction and workup steps that transforms a starting material into an end product ( He et al, 2021 ). The process of extracting these steps consists of two key tasks: chemical named entity recognition (NER) and event extraction (EE).…”

“…To identify the events, we explored rule-based, convolutional neural network (CNN)-based, and BERT-based methods. We evaluated our methods on the CLEF ChEMU-2020 dataset ( He et al, 2021 ) where we also participated in the challenge ( Mahendran et al, 2020 ). We evaluated each of the methods on their individual tasks (NER and EE) independently, and as an end-to-end system.…”