Graph neural network interatomic potential ensembles with calibrated aleatoric and epistemic uncertainty on energy and forces

Busk, Jonas; Schmidt, Mikkel N.; Winther, Ole; Vegge, Tejs; Jørgensen, Peter Bjørn

doi:10.1039/d3cp02143b

Cited by 4 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…107 This approach resulted in discovering new reactions or improved methods for existing reactions. Furthermore, emphasizing the uncertainty quantication of AI models was highlighted as a critical step, as rewarding areas of large uncertainty in active learning frameworks necessitates the quantication and understanding of the epistemic and aleatoric uncertainty of the models, 38,108,109 and the errors at each step.…”

Section: Going Beyond the Interpolative Nature Of Machine Learningmentioning

confidence: 99%

Accelerated chemical science with AI

Back,

Aspuru-Guzik,

Ceriotti

et al. 2024

Digital Discovery

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Going Beyond the Interpolative Nature Of Machine Learningmentioning

confidence: 99%

Accelerated chemical science with AI

Back,

Aspuru-Guzik,

Ceriotti

et al. 2024

Digital Discovery

Self Cite

View full text Add to dashboard Cite

show abstract

“…Busk et al used deep ensembles to obtain reliable uncertainty estimates for the property prediction of small organic molecules [24,25]. Thaler and coworkers observed that deep ensembles yield UQs of comparable quality to using stochastic gradient Markov chain Monte Carlo to sample the posterior [26].…”

Section: Uncertainty Model Architecturementioning

confidence: 99%

Uncertainty quantification by direct propagation of shallow ensembles

Kellner,

Ceriotti

2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Statistical learning algorithms provide a generally-applicable framework to sidestep time-consuming experiments, or accurate physics-based modeling, but they introduce a further source of error on top of the intrinsic limitations of the experimental or theoretical setup. Uncertainty estimation is essential to quantify this error, and to make application of data-centric approaches more trustworthy. To ensure that uncertainty quantification is used widely, one should aim for algorithms that are accurate, but also easy to implement and apply. In particular, including uncertainty quantification on top of an existing architecture should be straightforward, and add minimal computational overhead. Furthermore, it should be easy to manipulate or combine multiple machine-learning predictions, propagating uncertainty over further modeling steps. We compare several well-established uncertainty quantification frameworks against these requirements, and propose a practical approach, which we dub direct propagation of shallow ensembles, that provides a good compromise between ease of use and accuracy. We present benchmarks for generic datasets, and an in-depth study of applications to the field of atomistic machine learning for chemistry and materials. These examples underscore the importance of using a formulation that allows propagating errors without making strong assumptions on the correlations between different predictions of the model.

show abstract

Calibration in machine learning uncertainty quantification: Beyond consistency to target adaptivity

Pernot

2023

APL Machine Learning

View full text Add to dashboard Cite

Reliable uncertainty quantification (UQ) in machine learning (ML) regression tasks is becoming the focus of many studies in materials and chemical science. It is now well understood that average calibration is insufficient, and most studies implement additional methods for testing the conditional calibration with respect to uncertainty, i.e., consistency. Consistency is assessed mostly by so-called reliability diagrams. There exists, however, another way beyond average calibration, which is conditional calibration with respect to input features, i.e., adaptivity. In practice, adaptivity is the main concern of the final users of the ML-UQ method, seeking the reliability of predictions and uncertainties for any point in the feature space. This article aims to show that consistency and adaptivity are complementary validation targets and that good consistency does not imply good adaptivity. An integrated validation framework is proposed and illustrated with a representative example.

show abstract

Graph neural network interatomic potential ensembles with calibrated aleatoric and epistemic uncertainty on energy and forces

Abstract: A complete framework for training and recalibrating graph neural network ensemble models to produce accurate predictions of interatomic energy and forces with calibrated uncertainty estimates.

Cited by 4 publications

References 30 publications

Accelerated chemical science with AI

Accelerated chemical science with AI

Uncertainty quantification by direct propagation of shallow ensembles

Calibration in machine learning uncertainty quantification: Beyond consistency to target adaptivity

Contact Info

Product

Resources

About