Deep Neural Network Modeling for Big Data Weather Forecasting

Liu, James N. K.; Hu, Yong; He, Yulin; Chan, P. W.; Lai, Lucas

doi:10.1007/978-3-319-08254-7_19

Cited by 60 publications

(32 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Obviously, it is a very simple prediction method, which only takes two parameters into consideration. Recently, we find that deep learning can be used in prediction in some literatures [6], [7], [10]- [12]. Therefore, in this paper we try to study a prediction method for big data based on deep learning.…”

Section: Traditional Methods For Big Data Scale Predictionmentioning

confidence: 99%

“…But, whether the assumptions are correct or not may seriously affect the prediction accuracy. Recently, Liu et al [12] proposed a deep learning network with SAE model to simulate hourly weather data in 30 years. The model could learn weather features automatically from large scale weather data via layer-by-layer feature granulation and found statistical laws hidden in data.…”

Section: Related Workmentioning

confidence: 99%

“…In last few years, deep learning as one kind of artificial intelligence method attracted extensive attention from both academia and industry. At present, deep learning has been applied successfully in natural language processing [2], object detection and classification [3], [4], [34], [35], traffic flow prediction [5]- [9], [36], [37], disease prediction [10], [11], weather prediction [12], [13], and so on. The key of deep learning is to use neural network architectures with multiple or deeper layers.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Growth Scale Prediction of Big Data for Information Systems Based on a Deep Learning SAEP Method

Liu

Zeng

2020

IEEE Access

View full text Add to dashboard Cite

With the explosive growth of big data in various application areas, it is becoming very important for information management system to know the real-time growth change and the long-term increasing trend of big data. Thus, in this paper, we propose a big data growth scale prediction method based on deep learning intelligence. By analyzing the features of big data in various information systems, we summarize several basic data types which are ubiquitous in all kinds of information systems, and give the calculation method of each data growth scale time series. Considering the complex correlation among various types of data produced in information systems and the demand for data scale prediction, a deep learning prediction model, stacked autoencoders prediction (SAEP), is built. The previous time series prediction only considered one time series, but the SAEP can consider multiple time series simultaneously. We use data growth scale time series with different time interval as training sample sets to study features of big data in information systems, and then we can obtain the parameters of the SAEP model for the expected time interval. In order to solve the training problem of deep neural network, we also give a layer-wise training algorithm whose basic idea is to train all the hidden layers of the SAEP model layer by layer and then fine tune the parameters of the whole network. Repeated experiments show that the prediction performance of the SAEP model is stable and obviously better than the traditional exponential regression prediction method.

show abstract

Section: Traditional Methods For Big Data Scale Predictionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Growth Scale Prediction of Big Data for Information Systems Based on a Deep Learning SAEP Method

Liu

Zeng

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The enormously random variations of weather conditions lead to difficulties in the accuracy of the prediction [58]. Fortunately, different parameters of the weather can be predicted with significant accuracy by developing any of the latest models, including the ANN, Deep Neural Network (DNN), and LSTM [59][60][61].…”

Section: Related Workmentioning

confidence: 99%

Microgrid-Level Energy Management Approach Based on Short-Term Forecasting of Wind Speed and Solar Irradiance

2019

View full text Add to dashboard Cite

Background: The Distributed Energy Resources (DERs) are beneficial in reducing the electricity bills of the end customers in a smart community by enabling them to generate electricity for their own use. In the past, various studies have shown that owing to a lack of awareness and connectivity, end customers cannot fully exploit the benefits of DERs. However, with the tremendous progress in communication technologies, the Internet of Things (IoT), Big Data (BD), machine learning, and deep learning, the potential benefits of DERs can be fully achieved, although a significant issue in forecasting the generated renewable energy is the intermittent nature of these energy resources. The machine learning and deep learning models can be trained using BD gathered over a long period of time to solve this problem. The trained models can be used to predict the generated energy through green energy resources by accurately forecasting the wind speed and solar irradiance. Methods: We propose an efficient approach for microgrid-level energy management in a smart community based on the integration of DERs and the forecasting wind speed and solar irradiance using a deep learning model. A smart community that consists of several smart homes and a microgrid is considered. In addition to the possibility of obtaining energy from the main grid, the microgrid is equipped with DERs in the form of wind turbines and photovoltaic (PV) cells. In this work, we consider several machine learning models as well as persistence and smart persistence models for forecasting of the short-term wind speed and solar irradiance. We then choose the best model as a baseline and compare its performance with our proposed multiheaded convolutional neural network model. Results: Using the data of San Francisco, New York, and Los Vegas from the National Solar Radiation Database (NSRDB) of the National Renewable Energy Laboratory (NREL) as a case study, the results show that our proposed model performed significantly better than the baseline model in forecasting the wind speed and solar irradiance. The results show that for the wind speed prediction, we obtained 44.94%, 46.12%, and 2.25% error reductions in root mean square error (RMSE), mean absolute error (MAE), and symmetric mean absolute percentage error (sMAPE), respectively. In the case of solar irradiance prediction, we obtained 7.68%, 54.29%, and 0.14% error reductions in RMSE, mean bias error (MBE), and sMAPE, respectively. We evaluate the effectiveness of the proposed model on different time horizons and different climates. The results indicate that for wind speed forecast, different climates do not have a significant impact on the performance of the proposed model. However, for solar irradiance forecast, we obtained different error reductions for different climates. This discrepancy is certainly due to the cloud formation processes, which are very different for different sites with different climates. Moreover, a detailed analysis of the generation estimation and electricity bill reduction indicates that the proposed framework will help the smart community to achieve an annual reduction of up to 38% in electricity bills by integrating DERs into the microgrid. Conclusions: The simulation results indicate that our proposed framework is appropriate for approximating the energy generated through DERs and for reducing the electricity bills of a smart community. The proposed framework is not only suitable for different time horizons (up to 4 h ahead) but for different climates.

show abstract

“…In this paper, a deep‐learning method is proposed for the prediction of the ENSO and EQUINOO. Data‐driven machine learning and deep‐learning methods have shown good promise in addressing problems in climate sciences owing to the availability of huge amounts of climatic data (Liu et al ., ; Saha et al ., ). They have been found to be efficient in predicting the Indian monsoon for aggregate and regional parts (Saha et al ., , ).…”

Section: Introductionmentioning

confidence: 99%

Prediction of the ENSO and EQUINOO indices during June–September using a deep learning method

Saha

Nanjundiah

2019

Meteorological Applications

View full text Add to dashboard Cite

The Equatorial Indian Ocean Oscillation (EQUINOO) and El Niño Southern Oscillation (ENSO) are important climatic oscillations over the Indian and Pacific oceans influencing the inter‐annual variation of the Indian monsoon. The study of these indices, including their relationship and influence over various climatic phenomena, is the main focus in the literature. However, an attempt is made here to predict the indices for different temporal periods. Though ENSO prediction is established by many statistical and numerical models, the prediction of the EQUINOO index is not much studied. A deep‐learning method using an autoencoder is proposed for the prediction of the EQUINOO and ENSO. An autoencoder assists in feature learning. The learned features are ranked using linear and nonlinear correlation studies. This assists in identifying a set of potential predictors, used for indices prediction, with an ensemble of regression trees and decision forest models. Predictors identified by nonlinear correlation are observed to predict with better accuracy as compared with linear correlation. The predicted indices show high correlation against the observed values. The EQUINOO prediction is provided with a high lead of 7 months with a 0.88 correlation co‐efficient (p < 0.001) and the ENSO with a lead of 1 month with a 0.87 correlation co‐efficient (p < 0.001) between the observed and predicted indices. Moreover, the proposed method proves efficient in predicting the positive or negative index values with an appropriate sign. The ENSO prediction by the proposed approach is observed to be comparable with the existing models.

show abstract

Deep Neural Network Modeling for Big Data Weather Forecasting

Cited by 60 publications

References 37 publications

Growth Scale Prediction of Big Data for Information Systems Based on a Deep Learning SAEP Method

Growth Scale Prediction of Big Data for Information Systems Based on a Deep Learning SAEP Method

Microgrid-Level Energy Management Approach Based on Short-Term Forecasting of Wind Speed and Solar Irradiance

Prediction of the ENSO and EQUINOO indices during June–September using a deep learning method

Contact Info

Product

Resources

About