Transformer-based molecular optimization beyond matched molecular pairs

Abstract

Molecular optimization aims to improve the drug profile of a starting molecule. It is a fundamental problem in drug discovery but challenging due to (i) the requirement of simultaneous optimization of multiple properties and (ii) the large chemical space to explore. Recently, deep learning methods have been proposed to solve this task by mimicking the chemist’s intuition in terms of matched molecular pairs (MMPs). Although MMPs is a widely used strategy by medicinal chemists, it offers limited capability in terms of exploring the space of structural modifications, therefore does not cover the complete space of solutions. Often more general transformations beyond the nature of MMPs are feasible and/or necessary, e.g. simultaneous modifications of the starting molecule at different places including the core scaffold. This study aims to provide a general methodology that offers more general structural modifications beyond MMPs. In particular, the same Transformer architecture is trained on different datasets. These datasets consist of a set of molecular pairs which reflect different types of transformations. Beyond MMP transformation, datasets reflecting general structural changes are constructed from ChEMBL based on two approaches: Tanimoto similarity (allows for multiple modifications) and scaffold matching (allows for multiple modifications but keep the scaffold constant) respectively. We investigate how the model behavior can be altered by tailoring the dataset while using the same model architecture. Our results show that the models trained on differently prepared datasets transform a given starting molecule in a way that it reflects the nature of the dataset used for training the model. These models could complement each other and unlock the capability for the chemists to pursue different options for improving a starting molecule.

Can We Quickly Learn to “Translate” Bioactive Molecules with Transformer Models?

“…It is known that transformer models can learn general rules of composition of organic molecules without explicitly using information on their stability; ,, here, we extend such an approach to generating biologically active molecules. As examples of alternatives, we can mention ref where a separate LSTM model was trained to prioritize SMILES coming from a generative model, and refs , , , and where transformer models were trained to optimize computable properties (logD, solubility, clearance, or logP), but not biological activity of generated molecules against a given protein target. Note also that transfer learning or fine-tuning of a generative model on molecules active against the given target is not necessary, as we show in this work.…”

“…Machine learning (ML) has been actively used for drug design (for recent reviews, see refs − ). However, applications of ML specifically to hit expansion have shown limited demonstrated success. − A question arises whether this practice could be improved, and if so, what kind of ML models could better serve this purpose.…”

Transformer-based deep learning method for optimizing ADMET properties of lead compounds

“…Following the data preparation method of He et al, 16 the CHEMBL 24 dataset was first standardized using MolVS 25 and subsequently cleaned by implementing operations such as removing complexes, removing hydrogen atoms, disconnecting metals, running reionizer acid, applying normalization rules and keeping stereo structures. Cumming et al 26 summarized the filters used to find high quality compounds, such as ''rule of five'', AZFilters, quantitative estimate of drug-likeness (QED), etc.…”

“…15 The major difference with MMPA is that by adding property constraints to the input of the translation model, targeted modifications to the starting molecule, such as lowering the log D or increasing the clearance rate, can be achieved. [16][17][18] However, relative constraints (optimized properties -original properties) in the previous method may increase the learning difficulty of the model and reduce its optimization performance. Based on the above observations, we believe that the development of deep learning-based molecular optimization tools needs to address two aspects: (1) the development of higher accuracy ADMET prediction models; (2) the development of condition-generation models that satisfy multiple constraints simultaneously.…”

A Perspective on Explanations of Molecular Prediction Models

“…[108,109] MMPs are widely used in drug discovery and medicinal chemistry as they facilitate fast and easy understanding of structure-activity relationships. [110][111][112] Counterfactual and MMP analysis intersect if the structural change is associated with a drastic change in the properties. These MMPs are then counterfactual pairs.…”