Exploration of transferable and uniformly accurate neural network interatomic potentials using optimal experimental design

Abstract

Machine learning has been proven to have the potential to bridge the gap between the accuracy of ab initio methods and the efficiency of empirical force fields. Neural networks are one of the most frequently used approaches to construct high-dimensional potential energy surfaces. Unfortunately, they lack an inherent uncertainty estimation which is necessary for efficient and automated sampling through the chemical and conformational space to find extrapolative configurations. The identification of the latter is needed for the construction of transferable and uniformly accurate potential energy surfaces. In this paper, we propose an active learning approach that uses the estimated model’s output variance derived in the framework of the optimal experimental design. This method has several advantages compared to the established active learning approaches, e.g. Query-by-Committee, Monte Carlo dropout, feature and latent distances, in terms of the predictive power and computational efficiency. We have shown that the application of the proposed active learning scheme leads to transferable and uniformly accurate potential energy surfaces constructed using only a small fraction of data points. Additionally, it is possible to define a natural threshold value for the proposed uncertainty metric which offers the possibility to generate highly informative training data on-the-fly.

Exploring chemical and conformational spaces by batch mode deep active learning

“…Applying the GP posterior transformation to the last-layer kernel yields an equivalent AL method to the one based on OED. 19 However, in our experimental results, the application of random projections to the full gradient kernel leads to better uncertainty estimates than OED-uncertainties. Finally, various selection algorithms are introduced, ranging from the naive selection of multiple samples with the highest informativeness to more elaborated algorithms, which satisfy (1)-(3).…”

“…( 21), i.e. σ 2 ( E| D) = k ( S, S| D), which is equivalent to the results obtained in the OED framework, 19 if last-layer gradient features are used. A naive BMDAL approach would select N batch samples with the highest uncertainty.…”

“…A naive BMDAL approach would select N batch samples with the highest uncertainty. 19 However, the offdiagonal elements of kernel in Eq. ( 21) and Eq.…”

“…As an alternative approximation, we consider the use of last-layer gradients, whose extension to atomistic NNs is straightforward and has been used recently by some of us. 19 In summary, we propose random projections and last-layer gradients to efficiently approximate the full gradient kernel.…”

Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments

“…While our improved NN allows for convenient training on a given data set, it is especially useful when building the training data set on-the-fly, e.g., during a molecular dynamics (MD) simulation, or offline, drawing new training structures from an unlabeled data set. These workflows are referred to as learning-on-the-fly or active learning, ,− respectively. They reduce the number of required ab initio calculations but require frequent retraining of the model from updated training sets.…”