Molecular Transformer: A Model for Uncertainty-Calibrated Chemical Reaction Prediction

Abstract

Organic synthesis is one of the key stumbling blocks in medicinal chemistry. A necessary yet unsolved step in planning synthesis is solving the forward problem: Given reactants and reagents, predict the products. Similar to other work, we treat reaction prediction as a machine translation problem between simplified molecular-input line-entry system (SMILES) strings (a text-based representation) of reactants, reagents, and the products. We show that a multihead attention Molecular Transformer model outperforms all algorithms in the literature, achieving a top-1 accuracy above 90% on a common benchmark data set. Molecular Transformer makes predictions by inferring the correlations between the presence and absence of chemical motifs in the reactant, reagent, and product present in the data set. Our model requires no handcrafted rules and accurately predicts subtle chemical transformations. Crucially, our model can accurately estimate its own uncertainty, with an uncertainty score that is 89% accurate in terms of classifying whether a prediction is correct. Furthermore, we show that the model is able to handle inputs without a reactant–reagent split and including stereochemistry, which makes our method universally applicable.

1.4 The Future of Computing: Bits + Neurons + Qubits

“…Knowledge graph generation, simplification, and comparison technologies, as well as powerful graph inference methods, allow unprecedented fidelity in knowledge representation [64], and simplify how experts can consume the large volumes extracted. Moreover, machine learning-based surrogate models for physical systems [65] achieve more than 90% accuracy in solving complex chemical problems, assisting expert chemists in designing complex synthesis pathways. The combination of these technologies, based on "bits + neurons," has led to the development of new tools that permit domain experts to focus more upon outcomes, with the ultimate goal of speeding up groundbreaking new science and reducing the time-tomarket for innovative materials.…”

Transformer neural network for protein-specific de novo drug generation as a machine translation problem

“…The method has benefited from the recent progress in the neural machine translation field where the Transformer architecture demonstrated state-of-the-art results [34]. Recently the Transformer also exhibited very promising results in predicting the products of chemical reaction and retrosynthesis [56,57]. One of the key features of the Transformer is self-attention layers.…”

Predicting Retrosynthetic Pathways Using a Combined Linguistic Model and Hyper-Graph Exploration Strategy

“…Concurrently to rule-based systems, a wide range of AI approaches have been reported for retrosynthetic analysis [9,12], prediction of reaction outcomes [21][22][23][24][25][26] and optimization of reaction conditions [27]. All these AI models superseded rule-based methods in their potential of mimicking the human brain by learning chemistry from large data sets without human intervention.…”

“…Among the different AI approaches [39] those treating chemical reaction prediction as natural language (NL) problems [40] are becoming increasingly popular. They are currently state of the art in the forward reaction prediction realm, scoring an undefeated accuracy of more than 90% [22]. In the NL framework, chemical reactions are encoded as sentences using reaction SMILES [41] and the forward-or retro-reaction prediction is cast as a translation problem, using different types of neural machine translation architectures.…”

“…Transformer-based retrosynthesis: current status. Inspired by the success of the Molecular Transformer [22,42,43] for forward reaction prediction, a few retrosynthetic models based on the same architecture were reported shortly after [32,33,[35][36][37]. Zheng et al [32] proposed a template-free self-corrected retrosynthesis predictor built on the Transformer architecture.…”