Synthetic pre-training for neural-network interatomic potentials

Abstract

Machine learning (ML) based interatomic potentials have transformed the field of atomistic materials modelling. However, ML potentials depend critically on the quality and quantity of quantum-mechanical reference data with which they are trained, and therefore developing datasets and training pipelines is becoming an increasingly central challenge. Leveraging the idea of "synthetic" (artificial) data that is common in other areas of ML research, we here show that synthetic atomistic data, themselves obtained at scale with an existing ML potential, constitute a useful pre-training task for neural-network interatomic potential models. Once pre-trained with a large synthetic dataset, these models can be fine-tuned on a much smaller, quantum-mechanical one, improving numerical accuracy and stability in computational practice. We demonstrate feasibility for a series of equivariant graph-neural-network potentials for carbon, and we carry out initial experiments to test the limits of the approach.

Performance Assessment of Universal Machine Learning Interatomic Potentials: Challenges and Directions for Materials’ Surfaces

“…The combination of active learning strategies using cheap UIP trajectories to generate synthetic data sets for pretraining MLIPs can also greatly increase the efficiency of generating new data sets and expanding current ones. 27,34 Making use of already existing data sets, although at different fidelity levels, can be explored in multifidelity or multimodality approaches, recently shown to be beneficial and accelerating fine-tuning more than 100 ×. 35 Additionally, there is still plenty of exploration of potential gains related to the architectural design of the models.…”

“…In terms of efficiency, fine-tuning UIPs can greatly accelerate the training by incorporating alchemical transferable knowledge from the large bulk data set (in this case, MPtrj), thus requiring only a modest data set to achieve sufficient accuracy in specialized tasks. The combination of active learning strategies using cheap UIP trajectories to generate synthetic data sets for pretraining MLIPs can also greatly increase the efficiency of generating new data sets and expanding current ones. , Making use of already existing data sets, although at different fidelity levels, can be explored in multifidelity or multimodality approaches, recently shown to be beneficial and accelerating fine-tuning more than 100 × . Additionally, there is still plenty of exploration of potential gains related to the architectural design of the models.…”