Deep Generative Models Enable Navigation in Sparsely Populated Chemical Space

Abstract

Deep generative models are powerful tools for the exploration of chemical space, enabling the on-demand gener- ation of molecules with desired physical, chemical, or biological properties. However, these models are typically thought to require training datasets comprising hundreds of thousands, or even millions, of molecules. This per- ception limits the application of deep generative models in regions of chemical space populated by only a small number of examples. Here, we systematically evaluate and optimize generative models of molecules for low-data settings. We carry out a series of systematic benchmarks, training more than 5,000 deep generative models and evaluating over 2.6 billion generated molecules. We find that robust models can be learned from far fewer examples than has been widely assumed. We further identify strategies that dramatically reduce the number of molecules required to learn a model of equivalent quality, and demonstrate the application of these principles by learning models of chemical structures found in bacterial, plant, and fungal metabolomes. The structure of our experiments also allows us to benchmark the metrics used to evaluate generative models themselves. We find that many of the most widely used metrics in the field fail to capture model quality, but identify a subset of well-behaved metrics that provide a sound basis for model development. Collectively, our work provides a foundation for directly learning generative models in sparsely populated regions of chemical space.

Beam Search for Automated Design and Scoring of Novel ROR Ligands with Machine Intelligence**

“…Generative deep learning, [1, 2] that is, a class of machine learning models able to generate new data, can be applied to computationally design pharmacologically active compounds de novo [3–5] . Deep learning‐based molecular design algorithms can extract high‐level molecular features from “raw” molecular representations, [6–10] such as molecular graphs and the Simplified Molecular Input Line Entry System (SMILES, Figure 1 a), [11] potentially allowing them to access unexplored regions of the chemical space [12] . Previous studies showed that chemical language models (CLMs), [13, 14] in particular generative deep learning models trained on SMILES strings, can generate novel molecules with experimentally validated bioactivity [9, 15, 16] .…”

“…[ 3 , 4 , 5 ] Deep learning‐based molecular design algorithms can extract high‐level molecular features from “raw” molecular representations,[ 6 , 7 , 8 , 9 , 10 ] such as molecular graphs and the Simplified Molecular Input Line Entry System (SMILES, Figure 1 a ), [11] potentially allowing them to access unexplored regions of the chemical space. [12] Previous studies showed that chemical language models (CLMs),[ 13 , 14 ] in particular generative deep learning models trained on SMILES strings, can generate novel molecules with experimentally validated bioactivity. [ 9 , 15 , 16 ] CLMs have shown the ability to learn focused chemical features from small collections of template molecules by means of transfer learning, that is, a method to reuse previously learned knowledge on a new task for which the available data is scarce.…”

Beam‐Search zum automatisierten Entwurf und Scoring neuer ROR‐Liganden mithilfe maschineller Intelligenz**

“…Deep‐Learning‐basierte Algorithmen für das Moleküldesign können spezifische chemische Merkmale aus “rohen” Moleküldarstellungen, wie z. B. molekularen Graphen und dem Simplified Molecular Input Line Entry System (SMILES, Abbildung 1 a) [11] extrahieren, [6–10] was ihnen potenziell den Zugang zu unerforschten Regionen des chemischen Raums ermöglicht [12] . Frühere Studien haben gezeigt, dass chemische Sprachmodelle (Chemical Language Models, CLMs), [13, 14] insbesondere auf SMILES‐Strings trainierte generative Deep‐Learning‐Modelle, neuartige Moleküle mit experimentell validierter biologischer Aktivität generieren können [9, 15, 16] .…”