The rise and fall of artificial neural networks is well documented in the scientific literature of both computer science and computational chemistry. Yet almost two decades later, we are now seeing a resurgence of interest in deep learning, a machine learning algorithm based on multilayer neural networks. Within the last few years, we have seen the transformative impact of deep learning in many domains, particularly in speech recognition and computer vision, to the extent that the majority of expert practitioners in those field are now regularly eschewing prior established models in favor of deep learning models. In this review, we provide an introductory overview into the theory of deep neural networks and their unique properties that distinguish them from traditional machine learning algorithms used in cheminformatics. By providing an overview of the variety of emerging applications of deep neural networks, we highlight its ubiquity and broad applicability to a wide range of challenges in the field, including QSAR, virtual screening, protein structure prediction, quantum chemistry, materials design and property prediction. In reviewing the performance of deep neural networks, we observed a consistent outperformance against nonneural networks state-of-the-art models across disparate research topics, and deep neural network based models often exceeded the "glass ceiling" expectations of their respective tasks. Coupled with the maturity of GPU-accelerated computing for training deep neural networks and the exponential growth of chemical data on which to train these networks on, we anticipate that deep learning algorithms will be a valuable tool for computational chemistry. 3
IntroductionDeep Learning is the key algorithm used in the development of AlphaGo, a Deep learning is a machine learning algorithm, not unlike those already in use in various applications in computational chemistry, from computer-aided drug design to materials property prediction. 5-8 Amongst some of its more high profile achievements include the Merck activity prediction challenge in 2012, where a deep neural network not only won the competition and outperformed Merck's internal baseline model, but did so without having a single chemist or biologist in their team. In a repeated success by a different research team, deep learning models achieved top positions in the Tox21 toxicity prediction challenge issued by NIH in 2014. 9 The unusually stellar performance of deep learning models in both predicting activity and toxicity in these recent examples, originate from the unique characteristics that distinguishes deep learning from traditional machine learning algorithms.For those unfamiliar with the intricacies of machine learning algorithms, we will highlight some of the key differences between traditional (shallow) machine learning and deep learning.The simplest example of a machine learning algorithm would be the ubiquitous least-squares linear regression. In linear regression, the underlying nature of the model is known (linear in th...