Automatic recognition of complex magnetic regions on the Sun in SDO magnetogram images and prediction of flares: Techniques and results for the revised flare prediction program Flarecast

Steward, Graham; Lobzin, V. V.; Cairns, Iver H.; Li, Bo; Neudegg, David

doi:10.1002/2017sw001595

Cited by 9 publications

(5 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike dichotomous metrics like TSS and HSS, AUC summarizes detection performance over all possible false-positive rates and, in particular, does not depend on the threshold selected to convert probabilistic forecasts into binary decisions. Consequently, the AUC is not as useful in flare prediction, especially when stringent false-positive control is exercised (Steward et al 2017).…”

Section: Hssmentioning

confidence: 99%

Predicting Solar Flares Using CNN and LSTM on Two Solar Cycles of Active Region Data

Sun

Bobra

Wang

et al. 2022

ApJ

View full text Add to dashboard Cite

We consider the flare prediction problem that distinguishes flare-imminent active regions that produce an M- or X-class flare in the succeeding 24 hr, from quiet active regions that do not produce any flares within ±24 hr. Using line-of-sight magnetograms and parameters of active regions in two data products covering Solar Cycles 23 and 24, we train and evaluate two deep learning algorithms—a convolutional neural network (CNN) and a long short-term memory (LSTM)—and their stacking ensembles. The decisions of CNN are explained using visual attribution methods. We have the following three main findings. (1) LSTM trained on data from two solar cycles achieves significantly higher true skill scores (TSSs) than that trained on data from a single solar cycle with a confidence level of at least 0.95. (2) On data from Solar Cycle 23, a stacking ensemble that combines predictions from LSTM and CNN using the TSS criterion achieves a significantly higher TSS than the “select-best” strategy with a confidence level of at least 0.95. (3) A visual attribution method called “integrated gradients” is able to attribute the CNN’s predictions of flares to the emerging magnetic flux in the active region. It also reveals a limitation of CNNs as flare prediction methods using line-of-sight magnetograms: it treats the polarity artifact of line-of-sight magnetograms as positive evidence of flares.

show abstract

Section: Hssmentioning

confidence: 99%

Predicting Solar Flares Using CNN and LSTM on Two Solar Cycles of Active Region Data

Sun

Bobra

Wang

et al. 2022

ApJ

View full text Add to dashboard Cite

show abstract

Section: A5 Bom (Flarecast Bureau Of Meteorology Australia)mentioning

confidence: 99%

“…Different performance metrics inform on different performance aspects. This is discussed in Jolliffe & Stephenson (2012) and other references specifically with regards to flare forecasting in Bloomfield et al (2012); Barnes et al (2016); Kubo et al (2017); Steward et al (2017); Murray et al (2018). Hence, we present a number of metrics and evaluation tools, but for brevity we refer to any of the above references for the definitions of specific metrics 1 .…”

Section: Standard Metrics and Evaluation Toolsmentioning

confidence: 99%

“…The details of the probabilistic model are well described in Steward et al (2011Steward et al ( , 2017. Flarecast II (not yet published but results are submitted here) uses the SDO HMI magnetogram imagery analysis capability developed for the original Flarecast model (Steward et al 2017) plus prior flaring history, and adds a machine learning technique (logistic regression) to generate a probabilistic forecast. Variables that describe HMI B los magnetograms are selected to minimize Aikake's Information Criteria (AIC), and logistic regression is used to estimate the coefficients of the model, and then used to generate M+, X+, region and full-disk, probabilistic and categorical deterministic forecasts output for flaring activity over the next 24 hours.…”

Section: A4 Assa (Korean Space Weather Center)mentioning

confidence: 99%

See 1 more Smart Citation

A Comparison of Flare Forecasting Methods. II. Benchmarks, Metrics, and Performance Results for Operational Solar Flare Forecasting Systems

Leka

Park

Kusano

et al. 2019

ApJS

Self Cite

View full text Add to dashboard Cite

Solar flares are extremely energetic phenomena in our Solar System. Their impulsive, often drastic radiative increases, in particular at short wavelengths, bring immediate impacts that motivate solar physics and space weather research to understand solar flares to the point of being able to forecast them. As data and algorithms improve dramatically, questions must be asked concerning how well the forecasting performs; crucially, we must ask how to rigorously measure performance in order to critically gauge any improvements. Building upon earlier-developed methodology (Barnes et al. 2016, Paper I), international representatives of regional warning centers and research facilities assembled in 2017 at the Institute for Space-Earth Environmental Research, Nagoya University, Japan to -for the first time -directly compare the performance of operational solar flare forecasting methods. Multiple quantitative evaluation metrics are employed, with focus and discussion on evaluation methodologies given the restrictions of operational forecasting. Numerous methods performed consistently above the "no skill" level, although which method scored top marks is decisively a function of flare event definition and the metric used; there was no single winner. Following in this paper series we ask why the performances differ by examining implementation details (Leka et al. 2019, Paper III), and then we present a novel analysis method to evaluate temporal patterns of forecasting errors in (Park et al. 2019, Paper IV). With these works, this team presents a well-defined and robust methodology for evaluating solar flare forecasting methods in both research and operational frameworks, and today's performance benchmarks against which improvements and new methods may be compared.

show abstract

“…Recently, many articles related to solar flare occurrence forecasts have been published, which include deterministic forecasts as well as probabilistic forecasts. Examples are human-judged forecasts (Crown 2012;Devos et al 2014;Kubo et al 2017;Murray et al 2017), statistical methods (Wheatland 2005;Falconer et al 2011;Bloomfield et al 2012;McCloskey et al 2016;Steward et al 2017;Leka et al 2018), and machine learning forecasts (Bobra & Couvidat 2015;Muranushi et al 2015;Huang et al 2018;Nishizuka et al 2017Nishizuka et al , 2018. Many authors have assessed the performance of the forecast models.…”

Section: Introductionmentioning

confidence: 99%

Why do some probabilistic forecasts lack reliability?

Kubo

2019

J. Space Weather Space Clim.

View full text Add to dashboard Cite

In this work, we investigate the reliability of the probabilistic binary forecast. We mathematically prove that a necessary, but not sufficient, condition for achieving a reliable probabilistic forecast is maximizing the Peirce skill score (PS S ) at the threshold probability of the climatological base rate. The condition is confirmed by using artificially synthesized forecast-outcome pair data and previously published probabilistic solar flare forecast models. The condition gives a partial answer as to why some probabilistic forecast system lack reliability, because the system, which does not satisfy the proved condition, can never be reliable. Therefore, the proved condition is very important for the developers of a probabilistic forecast system. The result implies that those who want to develop a reliable probabilistic forecast system must adjust or train the system so as to maximize PS S near the threshold probability of the climatological base rate.

show abstract

Automatic recognition of complex magnetic regions on the Sun in SDO magnetogram images and prediction of flares: Techniques and results for the revised flare prediction program Flarecast

Cited by 9 publications

References 63 publications

Predicting Solar Flares Using CNN and LSTM on Two Solar Cycles of Active Region Data

Predicting Solar Flares Using CNN and LSTM on Two Solar Cycles of Active Region Data

A Comparison of Flare Forecasting Methods. II. Benchmarks, Metrics, and Performance Results for Operational Solar Flare Forecasting Systems

Why do some probabilistic forecasts lack reliability?

Contact Info

Product

Resources

About