Accurate approximations to density functionals have recently been obtained via machine learning (ML). By applying ML to a simple function of one variable without any random sampling, we extract the qualitative dependence of errors on hyperparameters. We find universal features of the behavior in extreme limits, including both very small and very large length scales, and the noise-free limit. We show how such features arise in ML models of density functionals. V C 2015 Wiley Periodicals, Inc.DOI: 10.1002/qua.24939
IntroductionMachine learning (ML) is a powerful data-driven method for learning patterns in high-dimensional spaces via induction. It provides a whole suite of tools for analyzing data, fitting highly nonlinear functions, and dimensionality reduction. [1] Given a set of training data, ML algorithms learn via induction to predict new data. ML methods have been developed within the areas of statistics and computer science, and have been applied to a huge variety of data, including neuroscience, image and text processing, and robotics. We are interested primarily in kernel ridge regression (KRR), which is one such standard method in ML. In general, the quality of the KRR learning model performance is highly dependent on the hyperparameters chosen and the size of the training data.ML has enjoyed widespread success in many fields, and has recently become popular as a tool in quantum chemistry and materials science, [2][3][4][5][6][7][8][9][10] as shown by the articles in this special issue. In many of these applications, many ab initio calculations are performed, and ML is applied to various properties of the results of these calculations. Our primary interest is in creating a much more intimate relation between ML and electronic structure calculations. Many such calculations use density functional theory (DFT), because of its favorable balance between accuracy and computational efficiency. But all such calculations rely on some approximation of an energy component as a functional of the electronic density. In particular, we wish to explore the applications of ML to the construction of density functionals, [11][12][13][14][15] which have focused so far on approximating the kinetic energy (KE) of noninteracting electrons. An accurate, general approximation to this could make orbital-free DFT a practical reality. However, the ML methods that have been developed are quite general and have not been tailored to account for specific details of the quantum problem. For example, it was found that KRR could yield excellent results for the KE functional, while never yielding accurate functional derivatives. [11] The development of methods for bypassing this difficulty has been important for ML in general. [14] In this context, ML provides a completely different way of thinking about electronic structure. The traditional ab initio approach [16] to electronic structure involves deriving carefully constructed approximations to solve the Schr€ odinger equation, based on physical intuition, exact conditions and asymptotic b...