Performance of Deep and Shallow Neural Networks, the Universal Approximation Theorem, Activity Cliffs, and QSAR

Abstract

Neural networks have generated valuable Quantitative Structure-Activity/Property Relationships (QSAR/QSPR) models for a wide variety of small molecules and materials properties. They have grown in sophistication and many of their initial problems have been overcome by modern mathematical techniques. QSAR studies have almost always used so-called "shallow" neural networks in which there is a single hidden layer between the input and output layers. Recently, a new and potentially paradigm-shifting type of neural network based on Deep Learning has appeared. Deep learning methods have generated impressive improvements in image and voice recognition, and are now being applied to QSAR and QSAR modelling. This paper describes the differences in approach between deep and shallow neural networks, compares their abilities to predict the properties of test sets for 15 large drug data sets (the kaggle set), discusses the results in terms of the Universal Approximation theorem for neural networks, and describes how DNN may ameliorate or remove troublesome "activity cliffs" in QSAR data sets.

Predicting the Performance of Organic Corrosion Inhibitors

“…Neural network and other machine learning methods can generate models quickly and effectively and require few or no assumptions to be made about the form of the mathematical relationship, as they are universal approximators [27,28]. The rise of deep learning algorithms within the last five years has stimulated the use of neural network and machine learning approaches substantially [4,5,29].…”

Generative Models for Artificially‐intelligent Molecular Design

“…One such re‐imagining of an older concept is in the burgeoning field of deep learning. Certain characteristics of this set of methods enable and encourage their nascent applications in the field of molecular design ,. Essentially, deep learning models are universal function estimators and hypothesis generators:…”

Identification of Alcoholism Based on Wavelet Renyi Entropy and Three‐Segment Encoded Jaya Algorithm

“…Scholars have used various classifiers in medical brain image analysis, such as decision tree, support vector machine [25], and naive Bayesian classifier. Nevertheless, the feedforward neural network (FNN) won remarkable success, because of the universal approximation theorem [26], which says the following. Suppose is a bounded and nonconstant continuous function.…”