A method to resolve the overfitting problem in recurrent neural networks for prediction of complex systems&amp;#x2019; behavior

Mahdaviani, Kaveh; Mazyar, H.; Majidi, Saeed; Saraee, Mohamad

doi:10.1109/ijcnn.2008.4634332

Cited by 12 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case, a copy of the information flowing from input to output is diverted back in the hidden layers. ENN was designed for voice processing problems (Li et al, 2019) and is similar to the FFNN except for the addition of the context layer (Tampelini, Boscarioli, Peres, & Sampaio, 2011) which stores a copy of the information to be provided to the hidden layers in the subsequent calculation steps (Mahdaviani, Mazyar, Majidi, & Saraee, 2008). Each hidden layers have its own context layer with the number of nodes equal to the number of nodes in the corresponding hidden layer.…”

Section: Fig 1 Morecambe Bay Model Domain and Bathymetry With Observa...mentioning

confidence: 99%

Coastal forecast through coupling of Deep Learning and hydro-morphodynamical modelling

Kumar

Leonardi

2022

Preprint

View full text Add to dashboard Cite

As climate-driven risks for the world's coastlines increase, understanding and predicting morphological changes as well as developing efficient systems for coastal forecast has become of the foremost importance for adaptation to climate change and informed coastal management choices. Artificial Intelligence, especially deep learning, is a powerful technology that has been rapidly evolving over the last couple of decades and can offer new means of analysis for the coastal science field. Yet, the potential of these technologies for coastal geomorphology remains relatively unexplored with respect to other scientific fields. This article investigates the use of Artificial Neural Networks and Bayesian Networks in combination with fully coupled hydrodynamics and morphological models (Delft3D) for predicting morphological changes and sediment transport along coastal systems. Two sets of deep learning models were tested, one set relying on localized modelling outputs or localized data sources and one set having reduced dependency from modeling outputs and, once trained, solely relying on boundary conditions and coastline geometry. The first set of models provides regression values greater than 0.95 and 0.86 for training and testing. The second set of reduced-dependency models provides regression values greater than 0.84 and 0.76 for training and testing. Both model types require a running time of the order of minutes, compared to the several hours of running times of the hydrodynamic models.Our results highlight the potential of deep learning and statistical models for coastal applications.

show abstract

Section: Fig 1 Morecambe Bay Model Domain and Bathymetry With Observa...mentioning

confidence: 99%

Coastal forecast through coupling of Deep Learning and hydro-morphodynamical modelling

Kumar

Leonardi

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The results suggest that smaller neural structures outperform the complex ones in fast fading channels equalization problem. These results may be explained by the overfitting problem in bigger structures [13,14], which present a greater number of parameters than the necessary. The fit of the parameters to track the signal, which vary rapidly, is a hard work in bigger structures, like the DFE-EKF or DFE-UKF.…”

Section: Performance Evaluationmentioning

confidence: 99%

Fast Fading Channel Neural Equalization Using Levenberg-Marquardt Training Algorithm and Pulse Shaping Filters

Mota¹,

Leal²,

Lima

2014

IJCNS

View full text Add to dashboard Cite

Artificial Neural Network (ANN) equalizers have been successfully applied to mitigate Inter symbolic Interference (ISI) due to distortions introduced by linear or nonlinear communication channels. The ANN architecture is chosen according to the type of ISI produced by fixed, fast or slow fading channels. In this work, we propose a combination of two techniques in order to minimize ISI yield by fast fading channels, i.e., pulse shape filtering and ANN equalizer. Levenberg-Marquardt algorithm is used to update the synaptic weights of an ANN comprise only by two recurrent perceptrons. The proposed system outperformed more complex structures such as those based on Kalman filtering approach.

show abstract

“…The network is trained (with possible overfitting) and processed afterwards. Such techniques include pruning, weight sharing, weight decay, ensemble neural networks, and complexity regularization [35,37,38]. Pruning is the process of eliminating nodes and connections from the trained network.…”

Section: R Andoniementioning

confidence: 99%

Extreme Data Mining: Inference from Small Datasets

Andonie

2010

INT J COMPUT COMMUN

View full text Add to dashboard Cite

Neural networks have been applied successfully in many fields. However, satisfactory results can only be found under large sample conditions. When it comes to small training sets, the performance may not be so good, or the learning task can even not be accomplished. This deficiency limits the applications of neural network severely. The main reason why small datasets cannot provide enough information is that there exist gaps between samples, even the domain of samples cannot be ensured. Several computational intelligence techniques have been proposed to overcome the limits of learning from small datasets. We have the following goals: i. To discuss the meaning of "small" in the context of inferring from small datasets. ii. To overview computational intelligence solutions for this problem. iii. To illustrate the introduced concepts with a real-life application.

show abstract

A method to resolve the overfitting problem in recurrent neural networks for prediction of complex systems’ behavior

Cited by 12 publications

References 8 publications

Coastal forecast through coupling of Deep Learning and hydro-morphodynamical modelling

Coastal forecast through coupling of Deep Learning and hydro-morphodynamical modelling

Fast Fading Channel Neural Equalization Using Levenberg-Marquardt Training Algorithm and Pulse Shaping Filters

Extreme Data Mining: Inference from Small Datasets

Contact Info

Product

Resources

About