When not to use machine learning: A perspective on potential and limitations

Abstract

The unparalleled success of artificial intelligence (AI) in the technology sector has catalyzed an enormous amount of research in the scientific community. It has proven to be a powerful tool, but as with any rapidly developing field, the deluge of information can be overwhelming, confusing, and sometimes misleading. This can make it easy to become lost in the same hype cycles that have historically ended in the periods of scarce funding and depleted expectations known as AI winters. Furthermore, although the importance of innovative, high-risk research cannot be overstated, it is also imperative to understand the fundamental limits of available techniques, especially in young fields where the rules appear to be constantly rewritten and as the likelihood of application to high-stakes scenarios increases. In this article, we highlight the guiding principles of data-driven modeling, how these principles imbue models with almost magical predictive power, and how they also impose limitations on the scope of problems they can address. Particularly, understanding when not to use data-driven techniques, such as machine learning, is not something commonly explored, but is just as important as knowing how to apply the techniques properly. We hope that the discussion to follow provides researchers throughout the sciences with a better understanding of when said techniques are appropriate, the pitfalls to watch for, and most importantly, the confidence to leverage the power they can provide. Graphical abstract

Accurate, Uncertainty-Aware Classification of Molecular Chemical Motifs from Multimodal X-ray Absorption Spectroscopy

“…However, such models are not "uncertainty aware", i.e., they do not produce a measure of confidence on any given training example. As ML methods are inherently interpolative procedures, 59 knowing when data drifts out-of-sample is of key importance. UQ is a set of statistical methods broadly classified as predictive methods for estimating the statistical uncertainty.…”

“…Therefore, it is important to understand the applicability of our ML models to experimental data. Above all, a good model performance on experimental data can only be obtained when they are sampled from the same distribution as their training set made of simulated data . Therefore, the outcome is dictated by the degree of agreement between theory and experiment, which strongly depends on the level of theory and approximations used in the simulation, such as the treatment of the core-hole final state effects, edge alignment, and the finite temperature effects.…”

Speciation of Nanocatalysts Using X-ray Absorption Spectroscopy Assisted by Machine Learning

“…In machine learning terminology, out-of-distribution prediction is an open problem. Nevertheless, it is imperative to reduce the gap between training data distribution and experimental data distribution …”

Current Practices and Continuing Needs in Thermophysical Properties for the Chemical Industry

“…It is an open question, however, whether the QDFT results correspond to anything experimentally reproducible. Carbone warns about this kind of question and other caveats related to machine learning …”

“…Carbone warns about this kind of question and other caveats related to machine learning. 113 As a complement to Table 1, Table 2 presents a summary of when current methods can be judged "sufficient." In no case should current methods be judged as sufficient for missioncritical applications.…”