The biggest contributor to global warming is energy production and use. Moreover, a push for electrical vehicle and other economic developments are expected to further increase energy use. To combat these challenges, electrical load forecasting is essential as it supports energy production planning and scheduling, assists with budgeting, and helps identify saving opportunities. Machine learning approaches commonly used for energy forecasting such as feedforward neural networks and support vector regression encounter challenges with capturing time dependencies. Consequently, this paper proposes Sequence to Sequence Recurrent Neural Network (S2S RNN) with Attention for electrical load forecasting. The S2S architecture from language translation is adapted for load forecasting and a corresponding sample generation approach is designed. RNN enables capturing time dependencies present in the load data and S2S model further improves time modeling by combining two RNNs: encoder and decoder. The attention mechanism alleviates the burden of connecting encoder and decoder. The experiments evaluated attention mechanisms with different RNN cells (vanilla, LSTM, and GRU) and with varied time horizons. Results show that S2S with Bahdanau attention outperforms other models. Accuracy decreases as forecasting horizon increases; however, longer input sequences do not always increase accuracy. INDEX TERMS Attention mechanism, gated recurrent units, GRU, load forecasting, long short-term memory, LSTM, recurrent neural networks, sequence-to-sequence networks. LJUBISA SEHOVAC (Student Member, IEEE) received the B.Sc. degree in applied math and M.E.Sc. degree in software engineering, collaborative specialization in artificial intelligence

Energy Consumption has been continuously increasing due to the rapid expansion of high-density cities, and growth in the industrial and commercial sectors. To reduce the negative impact on the environment and improve sustainability, it is crucial to efficiently manage energy consumption. Internet of Things (IoT) devices, including widely used smart meters, have created possibilities for energy monitoring as well as for sensor based energy forecasting. Machine learning algorithms commonly used for energy forecasting such as feedforward neural networks are not well-suited for interpreting the time dimensionality of a signal. Consequently, this paper uses Recurrent Neural Networks (RNN) to capture time dependencies and proposes a novel energy load forecasting methodology based on sample generation and Sequence-to-Sequence (S2S) deep learning algorithm. The S2S architecture that is commonly used for language translation was adapted for energy load forecasting. Experiments focus on Gated Recurrent Unit (GRU) based S2S models and Long Short-Term Memory (LSTM) based S2S models. All models were trained and tested on one building-level electrical consumption dataset, with five-minute incremental data. Results showed that, on average, the GRU S2S models outperformed LSTM S2S, RNN S2S, and Deep Neural Network models, for short, medium, and long-term forecasting lengths.

Smart meter popularity has resulted in the ability to collect big energy data and has created opportunities for large-scale energy forecasting. Machine Learning (ML) techniques commonly used for forecasting, such as neural networks, involve computationally intensive training typically with data from a single building or a single aggregated load to predict future consumption for that same building or aggregated load. With hundreds of thousands of meters, it becomes impractical or even infeasible to individually train a model for each meter. Consequently, this paper proposes Similarity-Based Chained Transfer Learning (SBCTL), an approach for building neural network-based models for many meters by taking advantage of already trained models through transfer learning. The first model is trained in a traditional way whereas all other models transfer knowledge from the existing models in a chain-like manner according to similarities between energy consumption profiles. A Recurrent Neural Network (RNN) was used as the base forecasting model, two initialization techniques were considered, and different similarity measures were explored. The experiments show that SBCTL achieves accuracy comparable to traditional ML training while taking only a fraction of time.INDEX TERMS Big data, deep learning, energy forecasting, gated recurrent units, recurrent neural network, smart meters, transfer learning.