A review of optical chemical structure recognition tools

Rajan, Kohulan; Brinkhaus, Henning Otto; Zielesny, Achim; Steinbeck, Christoph

doi:10.1186/s13321-020-00465-0

Cited by 61 publications

(73 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of optical chemical structure recognition has long been studied in computational chemistry [6][7][8][9][10][11][12] , and the current state of the art was recently summarized in a review paper. 13 Most of the published approaches up to 2019 were rule-based methods. Such systems typically work by first vectorizing the input image (maybe even first identifying the image within a pdf page).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Img2Mol - Accurate SMILES Recognition from Molecular Graphical Depictions

Clevert

Lê²,

Winter³

et al. 2021

Preprint

View full text Add to dashboard Cite

<p>Automatic recognition of the molecular content of a molecule’s graphical depiction is an extremely challenging problem that remains largely unsolved despite decades of research. Recent advances in neural machine translation enable the auto-encoding of molecular structures in a continuous vector space of fixed size (latent representation) with low reconstruction errors. In this paper, we present a fast and accurate model combining a deep convolutional neural network learning from molecule depictions and a pre-trained decoder that translates the latent representation into the SMILES representation of the molecules. This combination allows to precisely infer a molecular structure from an image. Our rigorous evaluation show that Img2Mol is able to correctly translate up to 88% of the molecular depictions into their SMILES representation. A pretrained version of Img2Mol is made publicly available on GitHub for non-commercial users.</p>

show abstract

Section: Related Workmentioning

confidence: 99%

“…on Japanese Patent Office (JPO) data, obtained from Rajan et al 13 . Note that this data set contains many textual labels, including Japanese characters, and irregular features, including line thickness variations.…”

Section: Jpo a Collection Of 365 Images And Molecule Descriptions Basedmentioning

confidence: 99%

Img2Mol - Accurate SMILES Recognition from Molecular Graphical Depictions

Clevert

Lê²,

Winter³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Over the course of the last three decades, there has been an active development in the field of Optical Chemical Structure Recognition (OCSR). OCSR is the translation of an image of a chemical structure into a machine-readable representation [ 4 ]. Most OCSR tools are only capable of processing images with pure chemical structure depictions.…”

Section: Introductionmentioning

confidence: 99%

DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature

et al. 2021

Self Cite

View full text Add to dashboard Cite

Chemistry looks back at many decades of publications on chemical compounds, their structures and properties, in scientific articles. Liberating this knowledge (semi-)automatically and making it available to the world in open-access databases is a current challenge. Apart from mining textual information, Optical Chemical Structure Recognition (OCSR), the translation of an image of a chemical structure into a machine-readable representation, is part of this workflow. As the OCSR process requires an image containing a chemical structure, there is a need for a publicly available tool that automatically recognizes and segments chemical structure depictions from scientific publications. This is especially important for older documents which are only available as scanned pages. Here, we present DECIMER (Deep lEarning for Chemical IMagE Recognition) Segmentation, the first open-source, deep learning-based tool for automated recognition and segmentation of chemical structures from the scientific literature. The workflow is divided into two main stages. During the detection step, a deep learning model recognizes chemical structure depictions and creates masks which define their positions on the input page. Subsequently, potentially incomplete masks are expanded in a post-processing workflow. The performance of DECIMER Segmentation has been manually evaluated on three sets of publications from different publishers. The approach operates on bitmap images of journal pages to be applicable also to older articles before the introduction of vector images in PDFs. By making the source code and the trained model publicly available, we hope to contribute to the development of comprehensive chemical data extraction workflows. In order to facilitate access to DECIMER Segmentation, we also developed a web application. The web application, available at https://decimer.ai, lets the user upload a pdf file and retrieve the segmented structure depictions.

show abstract

“…Automatic extraction of a molecule from an image of its 2D chemical structure to a machine-readable format, termed optical chemical structure recognition, first emerged in the 1990s. [13][14][15][16][17][18] These systems were developed with the intent of mining ChemDraw type diagrams in the chemical literature to utilize the wealth of largely untapped chemical information that lies within publications. Over the following decades, more complex systems were developed, often based on the principles of their predecessors.…”

Section: Introductionmentioning

confidence: 99%

“…Over the following decades, more complex systems were developed, often based on the principles of their predecessors. [18][19][20][21][22][23][24][25][26][27][28][29] OSRA was the first chemical structure recognition open-source software, allowing new programs to be developed by direct extension.…”

Section: Introductionmentioning

confidence: 99%

ChemPix: Automated Recognition of Hand-drawn Hydrocarbon Structures Using Deep Learning

Weir¹,

Thompson²,

Choi³

et al. 2021

Preprint

View full text Add to dashboard Cite

<p>Inputting molecules into chemistry software, such as quantum chemistry packages, currently requires domain expertise, expensive software and/or cumbersome procedures. Leveraging recent breakthroughs in machine learning, we develop ChemPix: an offline, hand-drawn hydrocarbon structure recognition tool designed to remove these barriers. A neural image captioning approach consisting of a convolutional neural network (CNN) encoder and a long short-term memory (LSTM) decoder learned a mapping from photographs of hand-drawn hydrocarbon structures to machine-readable SMILES representations. We generated a large auxiliary training dataset, based on RDKit molecular images, by combining image augmentation, image degradation and background addition. Additionally, a small dataset of ~600 hand-drawn hydrocarbon chemical structures was crowd-sourced using a phone web application. These datasets were used to train the image-to-SMILES neural network with the goal of maximizing the hand-drawn hydrocarbon recognition accuracy. By forming a committee of the trained neural networks, we achieved a nearly 10 percentage point improvement of the molecule recognition accuracy and were able to assign a confidence value for the prediction based on the number of agreeing votes. The top ensemble model achieved a hand-drawn hydrocarbon recognition accuracy of 77% for the first prediction and 86% if the top 3 predictions were considered; in over 50% of cases, the model was at least 97% confident in the prediction, making it a promising tool for real-world use cases.</p>

show abstract

A review of optical chemical structure recognition tools

Cited by 61 publications

References 41 publications

Img2Mol - Accurate SMILES Recognition from Molecular Graphical Depictions

Img2Mol - Accurate SMILES Recognition from Molecular Graphical Depictions

DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature

ChemPix: Automated Recognition of Hand-drawn Hydrocarbon Structures Using Deep Learning

Contact Info

Product

Resources

About