Machine learning (ML) provides novel and powerful ways of accurately and efficiently recognizing complex patterns, emulating nonlinear dynamics, and predicting the spatio-temporal evolution of weather and climate processes. Off-the-shelf ML models, however, do not necessarily obey the fundamental governing laws of physical systems, nor do they generalize well to scenarios on which they have not been trained. We survey systematic approaches to incorporating physics and domain knowledge into ML models and distill these approaches into broad categories. Through 10 case studies, we show how these approaches have been used successfully for emulating, downscaling, and forecasting weather and climate processes. The accomplishments of these studies include greater physical consistency, reduced training time, improved data efficiency, and better generalization. Finally, we synthesize the lessons learned and identify scientific, diagnostic, computational, and resource challenges for developing truly robust and reliable physics-informed ML models for weather and climate processes. This article is part of the theme issue ‘Machine learning for weather and climate modelling’.

We discuss improving forecasts of winds in the lower stratosphere using machine learning to postprocess the output of the European Centre for Medium‐Range Weather Forecasts (ECMWF) Integrated Forecast System. We postprocess global three‐dimensional predictions and demonstrate distilling the analog ensemble (AnEn) method into a deep neural network, which reduces postprocessing latency to near zero maintaining increased forecast skill. This approach reduces the error with respect to ECMWF high‐resolution deterministic prediction between 2–15% for wind speed and 15–25% for direction and is on par with ECMWF ensemble (ENS) forecast skill to hour 60. Verifying with Loon data from stratospheric balloons, AnEn has 20% lower error than ENS for wind speed and 15% for wind direction, despite significantly lower real‐time computational cost to ENS. Similar performance patterns are reported for probabilistic predictions, with larger improvements of AnEn with respect to ENS. We also demonstrate that AnEn generates a calibrated probabilistic forecast.

No abstract