Investigating the performance of neural network backpropagation algorithms for TEC estimations using South African GPS data

Abstract

Abstract. In this work, results obtained by investigating the application of different neural network backpropagation training algorithms are presented. This was done to assess the performance accuracy of each training algorithm in total electron content (TEC) estimations using identical datasets in models development and verification processes. Investigated training algorithms are standard backpropagation (SBP), backpropagation with weight delay (BPWD), backpropagation with momentum (BPM) term, backpropagation with chunkwise weight update (BPC) and backpropagation for batch (BPB) training. These five algorithms are inbuilt functions within the Stuttgart Neural Network Simulator (SNNS) and the main objective was to find out the training algorithm that generates the minimum error between the TEC derived from Global Positioning System (GPS) observations and the modelled TEC data. Another investigated algorithm is the MatLab based Levenberg-Marquardt backpropagation (L-MBP), which achieves convergence after the least number of iterations during training. In this paper, neural network (NN) models were developed using hourly TEC data (for 8 years: 2000-2007) derived from GPS observations over a receiver station located at Sutherland (SUTH) (32.38 • S, 20.81 • E), South Africa. Verification of the NN models for all algorithms considered was performed on both "seen" and "unseen" data. Hourly TEC values over SUTH for 2003 formed the "seen" dataset. The "unseen" dataset consisted of hourly TEC data for 2002 and 2008 over Cape Town (CPTN) (33.95 • S, 18.47 • E) and SUTH, respectively. The models' verification showed that all algorithms investigated provide comparable results statistically, but differ significantly in terms of time required to achieve convergence during input-output data training/learning. This paper therefore provides a guide to neural network users for choosing appropriate algorithms based on the availability of computation capabilities used for research.

Temporal solar irradiance variability analysis using neural networks

“…It is known that neural networks interpolate well within the input space, and therefore the network is expected to reproduce the data set that was used to train it with relatively good accuracy (Habarulema et al, 2007 ;Habarulema & McKinnell 2012 ;Tebabal et al, 2015). However, this model may not generalize well to new data that is outside the training set (Srivastava et al, 2014;Demuth & Beale, 2000)).…”

Storm Time Total Electron Content Modeling Over African Low‐Latitude and Midlatitude Regions

“…The difficulty in modeling TEC for the low-latitude ionosphere due to mixture of complex mechanisms happening in this region was also highlighted in different TEC modeling literatures Liu et al, 2013;Sur et al, 2015). Previous empirical TEC modeling studies mainly considered diurnal variation, seasonal variation, solar and geomagnetic activity representations as modeling inputs (Chen et al, 2015;Dabbakuti & Ratnam, 2016;Ercha et al, 2012;Habarulema et al, 2007Habarulema et al, , 2010Habarulema et al, , 2011Habarulema, McKinnell, Cilliers, & Opperman, 2009;Habarulema & McKinnell, 2012;Le et al, 2016;Mao et al, 2005Mao et al, , 2008Uwamahoro & Habarulema, 2015;Watthanasangmechai et al, 2012). The performances of models 1 and 2 in following TEC enhancements and depletions for some storms are very encouraging when compared to previous works where the failure of ANN, IRI, EOF, and other empirical models in following storm time TEC enhancement was noticed (Habarulema et al, 2010(Habarulema et al, , 2011Kumar et al, 2015;Mukhtarov et al, 2013;Olwendo et al, 2012;Uwamahoro & Habarulema, 2015).…”

“…This iteration process is repeated over and over until the error converges to the optimum value (Fausett, 1994;Haykin, 1994;Sur et al, 2015). The Leverberg-Marquardt backpropagation is the type of algorithm used during training because of its time saving advantage (Habarulema & McKinnell, 2012;Hagan et al, 1996).…”

Reconstruction of Storm‐Time Total Electron Content Using Ionospheric Tomography and Artificial Neural Networks: A Comparative Study Over the African Region

“…FFNNs with Levernberg-Marquardt backpropagation algorithm were used during training in the current work. Such type of configuration is preferred especially the training algorithm that is well known for its time saving while implementing the input-output mapping process (Jang et al, 1997) and has previously been applied to quiet and storm-time TEC modeling over the South African midlatitude region (Habarulema & McKinnell, 2012;Uwamahoro & Habarulema, 2015).…”