Memory augmented recurrent neural networks for de-novo drug design

Abstract

A recurrent neural network (RNN) is a machine learning model that learns the relationship between elements of an input series, in addition to inferring a relationship between the data input to the model and target output. Memory augmentation allows the RNN to learn the interrelationships between elements of the input over a protracted length of the input series. Inspired by the success of stack augmented RNN (StackRNN) to generate strings for various applications, we present two memory augmented RNN-based architectures: the Neural Turing Machine (NTM) and the Differentiable Neural Computer (DNC) for the de-novo generation of small molecules. We trained a character-level convolutional neural network (CNN) to predict the properties of a generated string and compute a reward or loss in a deep reinforcement learning setup to bias the Generator to produce molecules with the desired property. Further, we compare the performance of these architectures to gain insight to their relative merits in terms of the validity and novelty of the generated molecules and the degree of property bias towards the computational generation of de-novo drugs. We also compare the performance of these architectures with simpler recurrent neural networks (Vanilla RNN, LSTM, and GRU) without an external memory component to explore the impact of augmented memory in the task of de-novo generation of small molecules.

Toward Green AI: A Methodological Survey of the Scientific Literature

“…Alone, RNNs show a deficiency: their limited memory causes both computed and input data storage difficulties. Therefore, the availability of a higher quantity of memory for RNNs is crucial as it enables a stronger understanding of the relationships between input elements, as stressed by Suresh et al [39]. Furthermore, merging TM and RNNs allows for a new architecture that is memory augmented as RNNs own the equivalent infinite memory tape as TM.…”

“…By selecting data structures that use memory efficiently, it is expected to reduce the energy required for memory operations such as reading, writing, and moving data. For example, using a linked list instead of an array can save memory Harvard architecture [26] 1946 Poor It can improve performance in certain applications, but may also require more complex programming techniques High-Performance Computing [28], [29] 1960 Poor Due to their high processing power, HPC typically consumes substantial amounts of energy Quantum computers [30] [31] 1979 Medium Radical paradigmatic change in the concept of computing, although QC presents cooling issues Recurrent Neural Networks [38], [39] 1986 Poor RNNs are energy expensive and require a higher quantity of memory TrueNorth Architecture [47] 2014 N/A Architecture closely based on the human brain's organization and ANNs Evolutions of Turing machines (such as Neural Turing Machines [41], Neural Random Access Turing Machine [46])…”

Generative artificial intelligence in drug discovery: basic framework, recent advances, challenges, and opportunities

“…RNNs are often taught to create a single character at a time, with each character being informed by the characters that came before it in the molecular string. This allows them to be used to generate new molecular strings ( Suresh et al, 2022 ). Other popular CLM architectures, as seen earlier in this review, are a) VAE, constituted by an encoder that converts molecular strings to latent vectors and a decoder that converts latent vectors back to molecular strings, and b) GANs, constituted by a generator network that produces novel molecular strings and a discriminator network aiming to distinguish between the de novo designs and existing molecules.…”

The Third CACHE Challenge – Finding Ligands Targeting the Macrodomain of SARS-CoV-2 NSP3 Using AI-inspired and Knowledge-Based Approaches.

“…We implemented a deep recurrent neural network (RNN) 29 with long short-term memory (LSTM) cells for generation to design potential hit candidates for Mac1. RNNs have been instrumental in sequence modelling tasks, which predict or generate sequences of data (e.g., words making up sentences) where the temporal order or sequence of the data points is vital.…”