Can We Quickly Learn to “Translate” Bioactive Molecules with Transformer Models?

Abstract

Meaningful exploration of the chemical space of druglike molecules in drug design is a highly challenging task due to a combinatorial explosion of possible modifications of molecules. In this work, we address this problem with transformer models, a type of machine learning (ML) model originally developed for machine translation. By training transformer models on pairs of similar bioactive molecules from the public ChEMBL data set, we enable them to learn medicinal-chemistry-meaningful, context-dependent transformations of molecules, including those absent from the training set. By retrospective analysis on the performance of transformer models on ChEMBL subsets of ligands binding to COX2, DRD2, or HERG protein targets, we demonstrate that the models can generate structures identical or highly similar to most active ligands, despite the models having not seen any ligands active against the corresponding protein target during training. Our work demonstrates that human experts working on hit expansion in drug design can easily and quickly employ transformer models, originally developed to translate texts from one natural language to another, to "translate" from known molecules active against a given protein target to novel molecules active against the same target.

Open-Source Machine Learning in Computational Chemistry

“…MOLEcule Generation Using reinforcement Learning with Alternating Reward (MoleGuLAR) is an algorithm that proposes molecules for targeting specific protein binding sites using reinforcement learning . Using a ML transformer model that was trained on pairs of similar bioactive molecules, the transform-molecules algorithm can generate new molecules that ideally would have higher potency against a specific protein target . Kao et al developed QuMolGAN and related models to explore the use of quantum generative adversarial networks to create new molecules …”

“…313 Using a ML transformer model that was trained on pairs of similar bioactive molecules, the transform-molecules algorithm can generate new molecules that ideally would have higher potency against a specific protein target. 314 Kao et al developed QuMolGAN and related models to explore the use of quantum generative adversarial networks to create new molecules. 315 With a slightly different end goal, several algorithms approach the creation of new molecules by designing chemical linkers to combine fragments.…”

Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Design

“…Recently, advances in NLP have found surprising, strong results in the chemistry domain by training LLMs (Fabian et al, 2020;Chithrananda et al, 2020;NVIDIA Corporation, 2022;Tysinger et al, 2023) on string representations of molecules (Weininger, 1988;Weininger et al, 1989;Krenn et al, 2020;Cheng et al, 2023). To enable higher-level control over molecular design, multi-modal models (Edwards et al, 2021;Vall et al, 2021;Zeng et al, 2022;Xu and Wang, 2022;Su et al, 2022;Seidl et al, 2023;Xu et al, 2023;Zhao et al, 2023;Liu et al, 2023b) have been proposed.…”

SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design

“…Several efforts [18,11,77,66,50,75] show excellent results training on string representations of molecules [81,82,34,10]. Interest has also grown in multi-modal models [17,96] and multi-encoder models [16,76,85,71,42,67,87,97] with applications to chemistry and biology.…”