Feature Selection on a Flare Forecasting Testbed: A Comparative Study of 24 Methods

Yeoleka, Atharv; Patel, Sachin; Talla, Shreejaa; Puthucode, Krishna Rukmini; Ahmadzadeh, Azim; Sadykov, Viacheslav; Angryk, Rafal A.

doi:10.1109/icdmw53433.2021.00138

Cited by 6 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the figure, it is evident that the summary statistics from only a few magnetogram parameters tend to be chosen for a given forecast: ABSNJZH (absolute value of the net current helicity), SAVNCPP (sum of the absolute value of net current polarity), TOTUSJH (total unsigned current helicity), TOTBSQ (total magnitude of the Lorentz force), TOTPOT (total photospheric magnetic free energy density), TOTUSJZ (total unsigned vertical current), and USFLUX (total unsigned flux). These features align well with those highlighted in other work within the field (Bobra & Couvidat 2015;Yeolekar et al 2021;Zhang et al 2022).…”

supporting

confidence: 88%

See 1 more Smart Citation

Investigating Performance Trends of Simulated Real-time Solar Flare Predictions: The Impacts of Training Windows, Data Volumes, and the Solar Cycle

Goodwin,

Sadykov,

Martens

2024

ApJ

View full text Add to dashboard Cite

This study explores the behavior of machine-learning-based flare forecasting models deployed in a simulated operational environment. Using Georgia State University’s Space Weather Analytics for Solar Flares benchmark data set, we examine the impacts of training methodology and the solar cycle on decision tree, support vector machine, and multilayer perceptron performance. We implement our classifiers using three temporal training windows: stationary, rolling, and expanding. The stationary window trains models using a single set of data available before the first forecasting instance, which remains constant throughout the solar cycle. The rolling window trains models using data from a constant time interval before the forecasting instance, which moves with the solar cycle. Finally, the expanding window trains models using all available data before the forecasting instance. For each window, a number of input features (1, 5, 10, 25, 50, and 120) and temporal sizes (5, 8, 11, 14, 17, and 20 months) were tested. To our surprise, we found that, for a window of 20 months, skill scores were comparable regardless of the window type, feature count, and classifier selected. Furthermore, reducing the size of this window only marginally decreased stationary and rolling window performance. This implies that, given enough data, a stationary window can be chosen over other window types, eliminating the need for model retraining. Finally, a moderately strong positive correlation was found to exist between a model’s false-positive rate and the solar X-ray background flux. This suggests that the solar cycle phase has a considerable influence on forecasting.

show abstract

supporting

confidence: 88%

“…Ultimately, the main reason we settled on this feature selection approach over others is its proven success within SWAN-SF. Generally, F-values have been shown to provide reasonable insight into a feature's importance (Yeolekar et al 2021).…”

Section: Data Preprocessing Feature Selection and Hyperparameter Tuningmentioning

confidence: 99%

Investigating Performance Trends of Simulated Real-time Solar Flare Predictions: The Impacts of Training Windows, Data Volumes, and the Solar Cycle

Goodwin,

Sadykov,

Martens

2024

ApJ

View full text Add to dashboard Cite

show abstract

“…Furthermore, particularly some studies that used this dataset achieved significant results (Bobra and Couvidat, 2015;Bobra and Ilonidis, 2016;Raboonik et al, 2016;Nishizuka et al, 2017;Florios et al, 2018;Inceoglu et al, 2018;Ahmadzadeh et al, 2019;Hazra, Sardar, and Chowdhury, 2020;Ribeiro and Gradvohl, 2021;Yeolekar et al, 2021;Yi et al, 2021;Sinha et al, 2022).…”

Section: Methodsmentioning

confidence: 97%

Transformers as a classifier for solar flare time series: a comparative study

Ferreira,

Gradvohl,

Silva

et al. 2024

Preprint

View full text Add to dashboard Cite

Solar flares are violent and sudden eruptions that occur in the solar atmosphere and release energy in the form of radiation. They can affect technological systems on Earth and in its orbit, causing financial losses and damage to human life. Therefore, it is necessary to predict the occurrence of such flares to mitigate their effects. Specialized instruments gather data for solar activity monitoring. Hence, we can create prediction models using machine learning from this data. From an analysis of the literature, we noticed the prevalence of some algorithms, such as Multi-layer Perceptrons (MLP), Support Vector Machines (SVM), and Long Short-Term Memory (LSTM), which presented good results, mainly considering the True Skill Statistic (TSS) metric. In parallel, in 2017, a new deep learning-based neural network architecture called Transformers emerged. Researchers initially created it for natural language processing. However, Transformers were successfully employed in other domains, such as time series forecasting. Solar activity data is considered a time series due to its continuous capture over time. Consequently, we can employ Transformers to develop a solar flare forecast model. Considering a significant lack of work using Transformers for solar flare forecasting, we ran experiments to test the Transformers' viability and performance in solar flare forecast models. We created models using other algorithms (MLP, SVM, LSTM, Transformers) to investigate the Transformers' performance and compared them using accuracy, TSS, and Area Under the ROC Curve (AUC) metrics. We observed that the Transformers had superior performance compared to the other models. For instance, the Transformers' TSS metric average was 0.9, contrasting the other models' TSS=0.4. The difference was slightly smaller in AUC, where Transformers reached 0.9, and the others reached no more than 0.7. Therefore, we can use the Transformers to classify solar flare data and obtain superior results compared to other models. We also conducted experiments using different forms of data balancing, including unbalanced data, balanced with undersampling, oversampling, and SMOTE techniques. The MLP, SVM, and LSTM models showed significant improvements in balance, where the average TSS increased from 0.1 to 0.4. On the other hand, Transformers were not sensitive to data balancing, presenting the most stable TSS in all cases.

show abstract

CGAN-based synthetic multivariate time-series generation: a solution to data scarcity in solar flare forecasting

Chen

Kempton

Ahmadzadeh

et al. 2022

Neural Comput & Applic

View full text Add to dashboard Cite

Feature Selection on a Flare Forecasting Testbed: A Comparative Study of 24 Methods

Cited by 6 publications

References 36 publications

Investigating Performance Trends of Simulated Real-time Solar Flare Predictions: The Impacts of Training Windows, Data Volumes, and the Solar Cycle

Investigating Performance Trends of Simulated Real-time Solar Flare Predictions: The Impacts of Training Windows, Data Volumes, and the Solar Cycle

Transformers as a classifier for solar flare time series: a comparative study

CGAN-based synthetic multivariate time-series generation: a solution to data scarcity in solar flare forecasting

Contact Info

Product

Resources

About