Molecular Structure Extraction from Documents Using Deep Learning

Abstract

Chemical structure extraction from documents remains a hard problem due to both false positive identification of structures during segmentation and errors in the predicted structures. Current approaches rely on handcrafted rules and subroutines that perform reasonably well generally, but still routinely encounter situations where recognition rates are not yet satisfactory and systematic improvement is challenging. Complications impacting performance of current approaches include the diversity in visual styles used by various software to render structures, the frequent use of ad hoc annotations, and other challenges related to image quality, including resolution and noise. We here present end-to-end deep learning solutions for both segmenting molecular structures from documents and for predicting chemical structures from these segmented images. This deep learning-based approach does not require any handcrafted features, is learned directly from data, and is robust against variations in image quality and style. Using the deep-learning approach described herein we show that it is possible to perform well on both segmentation and prediction of low resolution images containing moderately sized molecules found in journal articles and patents.

Molecular representations in AI-driven drug discovery: a review and practical guide

“…Moreover, as the need for harvesting the large amounts of published data grows, the demand for methods for easily mining structures from papers and patent data is also growing. Optical Character Recognition (OCR) systems, relying on a variety of ML and probabilistic pattern recognition techniques, were created to translate 2D depictions of chemical structures to standard chemical representations [ 146 – 148 ]. Nonetheless, the development of OCR systems can be hindered by the images’ resolutions, the computational interpretations of chemical abbreviations, and the nature of the image representation, which can be embedded in text, in figures containing multiple structures, or in reaction pathways, and can be represented as either a skeletal formula or a Markush structure.…”

A review of optical chemical structure recognition tools

“…In 2019, Staker et al [ 41 ] presented a data-driven, deep learning based approach for OCSR called Molecular Structure Extraction from Documents Using Deep Learning (MSE-DUDL). The system uses two types of networks in the backend: a segmentation network and a structure prediction network.…”

Img2Mol - Accurate SMILES Recognition from Molecular Graphical Depictions

“…To the best of our knowledge, only two published methods were fully based on machine learning applied on the raw image data. MSE-DUDL 14 was published in 2019. It contains a segmentation network to extract molecule images from other components of the input page, coupled to a molecular recognition network.…”