Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning

Abstract

The fundamental goal of generative drug design is to propose optimized molecules that meet predefined activity, selectivity, and pharmacokinetic criteria. Despite recent progress, we argue that existing generative methods are limited in their ability to favorably shift the distributions of molecular properties during optimization. We instead propose a novel Reinforcement Learning framework for molecular design in which an agent learns to directly optimize through a space of synthetically accessible drug-like molecules. This becomes possible by defining transitions in our Markov decision process as chemical reactions and allows us to leverage synthetic routes as an inductive bias. We validate our method by demonstrating that it outperforms existing state-ofthe-art approaches in the optimization of pharmacologically relevant objectives, while results on multi-objective optimization tasks suggest increased scalability to realistic pharmaceutical design problems.

LibINVENT: Reaction-based Generative Scaffold Decoration for in Silico Library Design

“…Practical synthesis and chemical considerations should thus be taken into account when slicing the molecules to ensure that the reverse process (forward synthesis) is synthetically valid 25 . In their recent paper, Horwood and Noutahi 26 propose incorporating chemical synthesis routes directly into the model by designing a de novo generator based on chemical reactions. Given starting reactants, their model proposes drug-like molecules by selecting other appropriate reactants as well as specific reactions used to transform and connect the molecules into a resulting compound.…”

“…Without further regularisation or adjustments, these methods are known to suffer from high variance and instability 41 . In the case of molecular generation, however, the aim is to produce a large number of varied, interesting molecules 26 . This means that a certain level of variance is desirable to promote exploration of the chemical space and prevent mode collapse towards a single, high scoring solution.…”

Comparison of structure- and ligand-based scoring functions for deep generative models: a GPCR case study

“…Generative models can then be optimized to maximize this predicted value e.g. using reinforcement learning [ 7 , 8 , 20 , 21 ], Bayesian optimization [ 22 ] or particle swarm optimization [ 23 ]. Hence, a multitude of generative model methods exist, that can use none, one or multiple QSAR models or other external scoring functions to evaluate de novo molecules.…”

“…As a case study, we chose affinity for Dopamine Receptor D2 (DRD2). This receptor has a wealth of associated ligand bioactivity data available, and it has been commonly used in deep generative model publications before [ 7 , 21 , 22 , 29 , 44 ], thereby allowing any further comparison to different methods. DRD2 also has a publicly available X-ray crystal structure [ 45 ] in complex with Risperidone, thereby allowing use of molecular docking without the requirement of generating a homology model.…”

De Novo Design with Deep Generative Models Based on 3D Similarity Scoring

“…We have chosen a ligand-based design case study and the dopamine receptor D2 (DRD2) as the biological target of interest. This target has been widely used in de novo generation case studies by us [12], [23]- [25] and other groups [26]- [28]. As the generative model, we used REINVENT 2.0 [29] which is publicly available as open access software [30].…”