Chemical language models for molecular design

Abstract

In drug discovery, chemical language models (CLMs) originating from natural language processing offer new opportunities for molecular design. CLMs have been developed using recurrent neural network (RNN) or transformer architectures. For the predictive performance of RNN‐based encoder‐decoder frameworks and transformers, attention mechanisms play a central role. Among others, emerging application areas for CLMs include constrained generative modeling and the prediction of chemical reactions or drug‐target interactions. Since CLMs are applicable to any compound or target data that can be presented in a sequential format and tokenized, mappings of different types of sequences can be learned. For example, active compounds can be predicted from protein sequence motifs. Novel off‐the‐beat‐path applications can also be considered. For example, analogue series from medicinal chemistry can be perceived and represented as chemical sequences and extended with new compounds using CLMs. Herein, methodological features of CLMs and different applications are discussed.

Designing Small Molecule PI3Kγ Inhibitors: A Review of Structure-Based Methods and Computational Approaches

“…De Novo Drug Design by Chemical Language Models. A focused virtual chemical library for PI3Kγ was constructed by a chemical language model (CLM) 107,108 in two steps. In the first step, the CLM was pretrained with an autoregressive approach to capture features related to 839 674 patented compounds.…”

Extension of multi-site analogue series with potent compounds using a bidirectional transformer-based chemical language model

“…Such models can be derived to learn a variety of sequence-to-sequence mappings for diverse design tasks, giving rise to different types of chemical language models (CLMs). 6–14 Popular DNN architectures from NLP include recurrent neural networks (RNNs), which were first adapted for applications in chemistry, 4–7 and different transformer networks that are increasingly used for molecular design and de novo compound generation. 10–15 Different deep generative models including transformers and diffusion models have also been developed in structure-based drug design for the discovery and optimization of new active compounds.…”