Models based on neural networks and machine learning are seeing a rise in popularity in space physics. In particular, the forecasting of geomagnetic indices with neural network models is becoming a popular field of study. These models are evaluated with metrics such as the root-mean-square error (RMSE) and Pearson correlation coefficient. However, these classical metrics sometimes fail to capture crucial behavior. To show where the classical metrics are lacking, we trained a neural network, using a long short-term memory network, to make a forecast of the disturbance storm time index at origin time t with a forecasting horizon of 1 up to 6 h, trained on OMNIWeb data. Inspection of the model's results with the correlation coefficient and RMSE indicated a performance comparable to the latest publications. However, visual inspection showed that the predictions made by the neural network were behaving similarly to the persistence model. In this work, a new method is proposed to measure whether two time series are shifted in time with respect to each other, such as the persistence model output versus the observation. The new measure, based on Dynamical Time Warping, is capable of identifying results made by the persistence model and shows promising results in confirming the visual observations of the neural network's output. Finally, different methodologies for training the neural network are explored in order to remove the persistence behavior from the results.

Aims This paper presents a H2020 project aimed at developing the world’s most advanced space weather forecasting tool, combining the MagnetoHydroDynamic (MHD) solar wind and Coronal Mass Ejection (CME) evolution modelling with Solar Energetic Particle (SEP) transport and acceleration model(s). The EUHFORIA 2.0 project will address the geoeffectiveness of impacts and mitigation to avoid (part of the) damage, including that of extreme events, related to solar eruptions, solar wind streams, and SEPs, with particular emphasis on its application to forecast Geomagnetically Induced Currents (GICs) and radiation on geospace. Methods We will apply innovative methods and state-of-the-art numerical techniques to extend the recent heliospheric solar wind and CME propagation model EUHFORIA with two integrated key facilities that are crucial for improving its predictive power and reliability, namely 1) data-driven flux-rope CME models, and 2) physics-based, self-consistent SEP models for the acceleration and transport of particles along and across the magnetic field lines. This involves the novel coupling of advanced space weather models. In addition, after validating the upgraded EUHFORIA/SEP model, it will be coupled to existing models for GICs and atmospheric radiation transport models. This will result in a reliable prediction tool for radiation hazards from SEP events, affecting astronauts, passengers and crew in high-flying aircraft, and the impact of space weather events on power grid infrastructure, telecommunication, and navigation satellites. Finally, this innovative tool will be integrated into both the Virtual Space Weather Modeling Centre (VSWMC, ESA) and the space weather forecasting procedures at the ESA SSCC in Uccle (Belgium), so that it will be available to the space weather community and effectively used for improved predictions and forecasts of the evolution of CME magnetic structures and their impact on Earth. Results The results of the first six months of the EU H2020 project are presented here. These concern alternative coronal models, the application of adaptive mesh refinement techniques in the heliospheric part of EUHFORIA, alternative flux-rope CME models, evaluation of data-assimilation based on Karman filtering for the solar wind modelling, and a feasibility study of the integration of SEP models.

Over the last decades, international attempts have been made to develop realistic space weather prediction tools aiming to forecast the conditions on the Sun and in the interplanetary environment. These efforts have led to the development of appropriate metrics to assess the performance of those tools. Metrics are necessary to validate models, to compare different models, and to monitor the improvements to a certain model over time. In this work, we introduce dynamic time warping (DTW) as an alternative way of evaluating the performance of models and, in particular, of quantifying the differences between observed and modeled solar wind time series. We present the advantages and drawbacks of this method, as well as its application to Wind observations and EUHFORIA predictions at Earth. We show that DTW can warp sequences in time, aiming to align them with the minimum cost by using dynamic programming. It can be applied for the evaluation of modeled solar wind time series in two ways. The first calculates the sequence similarity factor, a number that provides a quantification of how good the forecast is compared to an ideal and a nonideal prediction scenario. The second way quantifies the time and amplitude differences between the points that are best matched between the two sequences. As a result, DTW can serve as a hybrid metric between continuous measurements (e.g., the correlation coefficient) and point-by-point comparisons. It is a promising technique for the assessment of solar wind profiles, providing at once the most complete evaluation portrait of a model.

Simulations of large-scale plasma systems are typically based on a fluid approximation approach. These models construct a moment-based system of equations that approximate the particle-based physics as a fluid, but as a result, they lack the small-scale physical processes available to fully kinetic models. Traditionally, empirical closure relations are used to close the moment-based system of equations, which typically approximate the pressure tensor or heat flux. The more accurate the closure relation, the stronger the simulation approaches kinetic-based results. In this paper, new closure terms are constructed using machine learning techniques. Two different machine learning models, a multi-layer perceptron and a gradient boosting regressor, synthesize a local closure relation for the pressure tensor and heat flux vector from fully kinetic simulations of a 2D magnetic reconnection problem. The models are compared to an existing closure relation for the pressure tensor, and the applicability of the models is discussed. The initial results show that the models can capture the diagonal components of the pressure tensor accurately and show promising results for the heat flux, opening the way for new experiments in multi-scale modeling. We find that the sampling of the points used to train both models plays a capital role in their accuracy.

No abstract