Comparing feature sets and machine-learning models for prediction of solar flares

Abstract

Context. Machine-learning methods for predicting solar flares typically employ physics-based features that have been carefully chosen by experts in order to capture the salient features of the photospheric magnetic fields of the Sun. Aims. Though the sophistication and complexity of these models have grown over time, there has been little evolution in the choice of feature sets, or any systematic study of whether the additional model complexity leads to higher predictive skill. Methods. This study compares the relative prediction performance of four different machine-learning based flare prediction models with increasing degrees of complexity. It evaluates three different feature sets as input to each model: a “traditional” physics-based feature set, a novel “shape-based” feature set derived from topological data analysis (TDA) of the solar magnetic field, and a combination of these two sets. A systematic hyperparameter tuning framework is employed in order to assure fair comparisons of the models across different feature sets. Finally, principal component analysis is used to study the effects of dimensionality reduction on these feature sets. Results. It is shown that simpler models with fewer free parameters perform better than the more complicated models on the canonical 24-h flare forecasting problem. In other words, more complex machine-learning architectures do not necessarily guarantee better prediction performance. In addition, it is found that shape-based feature sets contain just as much useful information as physics-based feature sets for the purpose of flare prediction, and that the dimension of these feature sets – particularly the shape-based one – can be greatly reduced without impacting predictive accuracy.

Investigating Performance Trends of Simulated Real-time Solar Flare Predictions: The Impacts of Training Windows, Data Volumes, and the Solar Cycle

“…Mathematically, DTs are constructed by minimizing the impurity of successive splits of the training data. This can be considered analogous to determining the feature and decision boundary that best separates two labeled distributions (Kingsford & Salzberg 2008;Kotsiantis 2013;Deshmukh et al 2023). In this work, we focus on optimizing two key hyperparameters of our DT model: the tree depth and the number of training samples needed for a split/leaf node to occur.…”

“…All data coming into a node are multiplied by its corresponding weight and summed together. The result is then transformed using an activation function and passed to the output, which connects to each node within the subsequent layer (Gardner & Dorling 1998;Deshmukh et al 2023). Typically, MLPs have three stages: a single input layer, a single output layer, and some arbitrarily large hidden layer sandwiched in between.…”

“…However, there is no guarantee that these models perform as well as their more complex counterparts. A previous study from Deshmukh et al (2023) found that, for flare forecasting, ML models of different complexities were quite comparable, but a similar investigation has yet to be done for a real-time forecasting environment. 3.…”

Persistent Homology Analysis for Solar Magnetograms

“…While the first factor is straightforward to measure (e.g., the sunspot area and total unsigned magnetic flux), persistent homology techniques may enable topological quantification of the latter two, which present greater challenges for conventional methods. Some studies have already delved into this concept; for example, Deshmukh et al (2023) employ a persistent homology analysis to explore the predictive capabilities of a machine-learning model to forecast solar flares based on the topological information extracted from solar magnetograms. However, they did not study the correspondence between the magnetic features and the topological information extracted from persistent homology, which is the main focus of this work.…”