Predicting chemical structure using reinforcement learning with a stack-augmented conditional variational autoencoder

Abstract

In this paper, a reinforcement learning model is proposed that can maximize the predicted binding affinity between a generated molecule and target proteins. The model used to generate molecules in the proposed model was the Stacked Conditional Variation AutoEncoder (Stack-CVAE), which acts as an agent in reinforcement learning so that the resulting chemical formulas have the desired chemical properties and show high binding affinity with specific target proteins. We generated 1000 chemical formulas using the chemical properties of sorafenib and the three target kinases of sorafenib. Then, we confirmed that Stack-CVAE generates more of the valid and unique chemical compounds that have the desired chemical properties and predicted binding affinity better than other generative models. More detailed analysis for 100 of the top scoring molecules show that they are novel ones not found in existing chemical databases. Moreover, they reveal significantly higher predicted binding affinity score for Raf kinases than for other kinases. Furthermore, they are highly druggable and synthesizable.

moBRCA-net: a breast cancer subtype classification framework based on multi-omics attention neural networks

“…For better generalization of moBRCA-net, we adopted a data augmentation based on the deep generative model to enlarge the training dataset size. Several recent papers have shown that conditional variational autoencoder (CVAE)-based data generation for certain minority classes in the imbalanced dataset improved the classification performance in various domain tasks such as respiratory disease classification [ 41 ], temporal pattern prediction based on electronic health records [ 42 ], and prediction of chemical structure based on the chemical properties [ 43 ]. We constructed a conditional variational autoencoder (CVAE) composed of two-layered encoder and decoder, which estimates the conditional distribution with latent variables and data, and generates samples for specified breast cancer subtype.…”

Living Conditions and Support Measures for Immigrant Youths in Yeonsu-gu, Incheon

“…Numerous studies have tackled the RL problem by defining an action space, a state, a policy, and an environment. The action space composed of symbol sets that represent molecular structures, a state space made up of symbol substrings, a policy for predicting the next appropriate symbol (action) to append to the current substring (state) up to a certain length, and an environment that evaluates the completed string, providing rewards based on its properties [10,11,16,18]. Policies employ deep neural network models, such as RNNs, to deal with string-based molecular structures.…”

“…To address these challenges, many studies have used the simplified molecular-input line-entry system (SMILES), self-referencing embedded strings (SELFIES) [7,8], and graph-based representation methods [9] in training deep molecular generative models. Researchers have exploited various deep generative models, including recurrent neural networks (RNNs) [10,11], transformers [12], and graph neural networks (GNNs) [9], to efficiently handle those string-based or graph-based molecular data [13,14]. Furthermore, Bayesian optimization [9,15] and reinforcement learning (RL) techniques [10,16,17] have been exploited for deep molecular generative models to create molecules with desired chemical properties.…”

“…Many studies have demonstrated the effectiveness of molecular structure generation strategies using RL for optimizing various molecular structure properties [10,11,16,17,18]. The typical RL configuration in AI-based drug design consists of two components: an agent that generates molecular structures and an environment that evaluate the generated molecules.…”

“…If a generated molecular structure possesses desired target properties, such as a high quantitative estimate of drug-likeness (QED), the agent receives a high reward as feedback from the environment. These property-constrained RL methods are effective in fine-tuning a pre-trained molecular generative model so that it can generate a large number of hit molecules [10,11,16,17,18]. However, owing to the vast size of the chemical space, it is challenging for an agent to perform efficient exploration, resulting in failure to find an optimal policy for the desired generation of molecular structures [14,19,20].…”