On regularization properties of artificial datasets for deep learning

Antczak, Karol

doi:10.5604/01.3001.0013.6599

Cited by 3 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This included L1 (i.e., least absolute deviation) and L2 (i.e., least square error) applied together with the dropout. It is imperative to mention that the role of L1 and L2 penalization type parameters is to minimize the sum of the absolute differences and the sum of the square of the differences between the forecasted and target PPFD values, respectively [107][108][109]. Also, the addition of a regularization to the loss is to encourage smooth network mapping in the DL network, particularly by penalizing the large parameters values to reduce the level of nonlinearity in the network models [110,111].…”

Section: Common Hyperparameters For Deep Learning (Dl) Modelsmentioning

confidence: 99%

Forecasting Photosynthetic Photon Flux Density Under Cloud Effects: Novel Predictive Model Using Convolutional Neural Network Integrated With Long Short-term Memory Network

Deo

Grant

Webb

et al. 2021

Preprint

View full text Add to dashboard Cite

Forecast models of solar radiation incorporating cloud effects are useful tools to evaluate the impact of stochastic behaviour of cloud movement, real-time integration of photovoltaic energy in power grids, skin cancer and eye disease risk minimisation through solar ultraviolet (UV) index prediction and bio-photosynthetic processes through the modelling of solar photosynthetic photon flux density (PPFD). This research has developed deep learning hybrid model (i.e., CNN-LSTM) to factor in role of cloud effects integrating the merits of convolutional neural networks with long short-term memory networks to forecast near real-time (i.e., 5-minute) PPFD in a sub-tropical region Queensland, Australia. The prescribed CLSTM model is trained with real-time sky images that depict stochastic cloud movements captured through a Total Sky Imager (TSI-440) utilising advanced sky image segmentation to reveal cloud chromatic features into their statistical values, and to purposely factor in the cloud variation to optimise the CLSTM model. The model, with its competing algorithms (i.e., CNN, LSTM, deep neural network, extreme learning machine and multivariate adaptive regression spline), are trained with 17 distinct cloud cover inputs considering the chromaticity of red, blue, thin, and opaque cloud statistics, supplemented by solar zenith angle (SZA) to predict short-term PPFD. The models developed with cloud inputs yield accurate results, outperforming the SZA-based models while the best testing performance is recorded by the objective method (i.e., CLSTM) tested over a 7-day measurement period. Specifically, CLSTM yields a testing performance with correlation coefficient r = 0.92, root mean square error RMSE = 210.31 μ mol of photons m-2 s-1, mean absolute error MAE = 150.24 μ mol of photons m-2 s-1, including a relative error of RRMSE = 24.92% MAPE = 38.01%, and Nash Sutcliffe’s coefficient ENS = 0.85, and Legate & McCabe’s Index LM = 0.68 using cloud cover in addition to the SZA as an input. The study shows the importance of cloud inclusion in forecasting solar radiation and evaluating the risk with practical implications in monitoring solar energy, greenhouses and high-value agricultural operations affected by stochastic behaviour of clouds. Additional methodological refinements such as retraining the CLSTM model for hourly and seasonal time scales may aid in the promotion of agricultural crop farming and environmental risk evaluation applications such as predicting the solar UV index and direct normal solar irradiance for renewable energy monitoring systems.

show abstract

Section: Common Hyperparameters For Deep Learning (Dl) Modelsmentioning

confidence: 99%

Forecasting Photosynthetic Photon Flux Density Under Cloud Effects: Novel Predictive Model Using Convolutional Neural Network Integrated With Long Short-term Memory Network

Deo

Grant

Webb

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This included L1 (i.e., least absolute deviation) and L2 (i.e., least square error) applied together with the dropout. It is imperative to mention that the role of L1 and L2 penalization type parameters is to minimize the sum of the absolute differences and the sum of the square of the differences between the forecasted and target PPFD values, respectively(Ayinde and Zurada 2017;Sato et al 2018;Antczak 2019). Also, the addition of a regularization to the loss is to encourage smooth network mapping in the DL network, particularly by penalizing the large parameters values to reduce the level of nonlinearity in the network models(Jaiswal et al 2018;Byrd and Lipton 2019).…”

mentioning

confidence: 99%

Forecasting solar photosynthetic photon flux density under cloud cover effects: novel predictive model using convolutional neural network integrated with long short-term memory network

Deo

Grant

Webb

et al. 2022

Stoch Environ Res Risk Assess

View full text Add to dashboard Cite

Forecast models of solar radiation incorporating cloud effects are useful tools to evaluate the impact of stochastic behaviour of cloud movement, real-time integration of photovoltaic energy in power grids, skin cancer and eye disease risk minimisation through solar ultraviolet (UV) index prediction and bio-photosynthetic processes through the modelling of solar photosynthetic photon flux density (PPFD). This research has developed deep learning hybrid model (i.e., CNN-LSTM) to factor in role of cloud effects integrating the merits of convolutional neural networks with long short-term memory networks to forecast near real-time (i.e., 5-min) PPFD in a sub-tropical region Queensland, Australia. The prescribed CLSTM model is trained with real-time sky images that depict stochastic cloud movements captured through a total sky imager (TSI-440) utilising advanced sky image segmentation to reveal cloud chromatic features into their statistical values, and to purposely factor in the cloud variation to optimise the CLSTM model. The model, with its competing algorithms (i.e., CNN, LSTM, deep neural network, extreme learning machine and multivariate adaptive regression spline), are trained with 17 distinct cloud cover inputs considering the chromaticity of red, blue, thin, and opaque cloud statistics, supplemented by solar zenith angle (SZA) to predict short-term PPFD. The models developed with cloud inputs yield accurate results, outperforming the SZA-based models while the best testing performance is recorded by the objective method (i.e., CLSTM) tested over a 7-day measurement period. Specifically, CLSTM yields a testing performance with correlation coefficient r = 0.92, root mean square error RMSE = 210.31 μ mol of photons m−2 s−1, mean absolute error MAE = 150.24 μ mol of photons m−2 s−1, including a relative error of RRMSE = 24.92% MAPE = 38.01%, and Nash Sutcliffe’s coefficient ENS = 0.85, and Legate and McCabe’s Index LM = 0.68 using cloud cover in addition to the SZA as an input. The study shows the importance of cloud inclusion in forecasting solar radiation and evaluating the risk with practical implications in monitoring solar energy, greenhouses and high-value agricultural operations affected by stochastic behaviour of clouds. Additional methodological refinements such as retraining the CLSTM model for hourly and seasonal time scales may aid in the promotion of agricultural crop farming and environmental risk evaluation applications such as predicting the solar UV index and direct normal solar irradiance for renewable energy monitoring systems.

show abstract

Deep learning in electron microscopy

Ede

2021

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

show abstract

On regularization properties of artificial datasets for deep learning

Cited by 3 publications

References 9 publications

Forecasting Photosynthetic Photon Flux Density Under Cloud Effects: Novel Predictive Model Using Convolutional Neural Network Integrated With Long Short-term Memory Network

Forecasting Photosynthetic Photon Flux Density Under Cloud Effects: Novel Predictive Model Using Convolutional Neural Network Integrated With Long Short-term Memory Network

Forecasting solar photosynthetic photon flux density under cloud cover effects: novel predictive model using convolutional neural network integrated with long short-term memory network

Deep learning in electron microscopy

Contact Info

Product

Resources

About