Transformer Neural Network for Structure Constrained Molecular Optimization

Abstract

Finding molecules with a desirable balance of multiple properties is a main challenge in drug discovery. Here, we focus on the task of molecular optimization, where a starting molecule with promising properties needs to be further optimized towards the desirable properties. Typically, chemists would apply chemical transformations to the starting molecule based on their intuition. A widely used strategy is the concept of matched molecular pairs where two molecules differ by a single transformation. In particular, a chemist would be interested in keeping one part of the starting molecule (core) constant, while substituting the other part (R-group), to optimize the starting molecule towards desirable properties. Motivated by this, we train a Transformer model, Transformer-R, to generate R-groups given the starting molecule (with its core and R-group specified) and the specified desirable properties. The generated R-groups will be attached to the core to form the final molecules, which are guaranteed to keep the core of interest and are expected to satisfy the desirable properties in the input. Our model could accelerate the process of optimizing antiviral drug candidates in terms of various properties of interest, e.g. pharmacokinetics.

Chemformer: a pre-trained transformer for computational chemistry

“…The USPTO-50K [35] dataset, which contains approximately 50 000 reactions, was used to benchmark Chemformer on the retrosynthesis prediction task. Molecular optimisation aims to improve the property profile of a starting molecule towards desirable molecular properties [6,7]. The dataset [6] for the molecular optimisation task consists of a set of matched molecular pairs (MMPs) extracted from ChEMBL [36], together with the property changes of the MMPs.…”

“…In particular, we examine the performance of all three pre-training tasks with base Chemformer models, after fine-tuning for 100 epochs, along with a Chemformer-Large model (pre-trained on the combined task) fine-tuned for 80 epochs. For the Transformer [6] and Transformer-R [7] benchmarks we use the published models, but examine only top-1 performance. From the table we can see that, while all Chemformer models perform strongly in comparison to existing benchmarks, the smaller Chemformer models outperform the larger on the percentage of desirable molecules generated.…”

“…Recent years have witnessed an explosion in research applying neural network models to cheminformatics tasks. Sequence-to-sequence models, such as the Transformer [1] and models based on the recurrent neural network architecture [2,3], are well suited to tasks such as direct reaction prediction [4,5], retrosynthesis prediction [5] and molecular optimisation [6,7]. Applying molecules encoded using the simplified molecular line entry system (SMILES) [8] to the Transformer model has produced state-of-the-art results in benchmark datasets for these tasks [5][6][7].…”

“…Sequence-to-sequence models, such as the Transformer [1] and models based on the recurrent neural network architecture [2,3], are well suited to tasks such as direct reaction prediction [4,5], retrosynthesis prediction [5] and molecular optimisation [6,7]. Applying molecules encoded using the simplified molecular line entry system (SMILES) [8] to the Transformer model has produced state-of-the-art results in benchmark datasets for these tasks [5][6][7]. Transformers have also been successfully applied to discriminative tasks, such as biological activity prediction (virtual screening) [9] and molecular property prediction (QSAR modelling) [9][10][11][12][13][14][15].…”

Dual-view Molecule Pre-training

“…Maziarka et al [37] improved self-attention with inter-atomic distances and molecular graph structure. In addition, there are some works about SMILES-based molecule generation [47,41] and optimization [18].…”