Short-Term Load Forecasting Based on the Transformer Model

Zhao, Zezheng; Xia, Chunqiu; Chi, Lian; Chang, Xiaomin; Li, Wei; Yang, Ting; Zomaya, Albert Y.

doi:10.3390/info12120516

Cited by 35 publications

(14 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wu et al [39] previously presented an adaptation of the transformer model for time series predictions; however, they did so to predict influenza cases rather than load forecasting and did not include many contextual features in their work. Zhao et al [40] presented a transformer implementation for load forecasting tasks, however the contextual features were handled outside of the transformer and therefore added another processing complexity layer to the solution. Additionally, their work provided little details in terms of implementation, and the evaluation was conducted on a single data stream and limited to one-day-ahead predictions.…”

Section: Related Workmentioning

confidence: 99%

“…These results are also highlighted in Figure 8b. Zhao et al [40] also employed the transformer for load forecasting; however, they used similar days' load data as the input for the transformer, while our approach uses previous load readings together with contextual features. Consequently, in contrast to the approach proposed by Zhao et al [40], our approach is flexible with regard to the number of lag readings considered for the forecasting [40], as demonstrated in experiments.…”

Section: Overall Performancementioning

confidence: 99%

“…Zhao et al [40] also employed the transformer for load forecasting; however, they used similar days' load data as the input for the transformer, while our approach uses previous load readings together with contextual features. Consequently, in contrast to the approach proposed by Zhao et al [40], our approach is flexible with regard to the number of lag readings considered for the forecasting [40], as demonstrated in experiments. Moreover, Zhao et al handle contextual features outside of the transformer and employ a transformer with six encoders and six decoders, which increases computational complexity.…”

Section: Overall Performancementioning

confidence: 99%

See 2 more Smart Citations

Transformer-Based Model for Electrical Load Forecasting

2022

View full text Add to dashboard Cite

Amongst energy-related CO2 emissions, electricity is the largest single contributor, and with the proliferation of electric vehicles and other developments, energy use is expected to increase. Load forecasting is essential for combating these issues as it balances demand and production and contributes to energy management. Current state-of-the-art solutions such as recurrent neural networks (RNNs) and sequence-to-sequence algorithms (Seq2Seq) are highly accurate, but most studies examine them on a single data stream. On the other hand, in natural language processing (NLP), transformer architecture has become the dominant technique, outperforming RNN and Seq2Seq algorithms while also allowing parallelization. Consequently, this paper proposes a transformer-based architecture for load forecasting by modifying the NLP transformer workflow, adding N-space transformation, and designing a novel technique for handling contextual features. Moreover, in contrast to most load forecasting studies, we evaluate the proposed solution on different data streams under various forecasting horizons and input window lengths in order to ensure result reproducibility. Results show that the proposed approach successfully handles time series with contextual data and outperforms the state-of-the-art Seq2Seq models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Overall Performancementioning

confidence: 99%

Section: Overall Performancementioning

confidence: 99%

See 1 more Smart Citation

Transformer-Based Model for Electrical Load Forecasting

2022

View full text Add to dashboard Cite

show abstract

“…In [16], an MTLF model based on the neural network approach is presented without any climatic information. The transformer models are suggested in [17,18] for load forecasting.…”

Section: Literature Reviewmentioning

confidence: 99%

A New Deep Learning Restricted Boltzmann Machine for Energy Consumption Forecasting

Tian

Firouzi

et al. 2022

Sustainability

View full text Add to dashboard Cite

A key issue in the desired operation and development of power networks is the knowledge of load growth and electricity demand in the coming years. Mid-term load forecasting (MTLF) has an important rule in planning and optimal use of power systems. However, MTLF is a complicated problem, and a lot of uncertain factors and variables disturb the load consumption pattern. This paper presents a practical approach for MTLF. A new deep learning restricted Boltzmann machine (RBM) is proposed for modelling and forecasting energy consumption. The contrastive divergence algorithm is presented for tuning the parameters. All parameters of RBMs, the number of input variables, the type of inputs, and also the layer and neuron numbers are optimized. A statistical approach is suggested to determine the effective input variables. In addition to the climate variables, such as temperature and humidity, the effects of other variables such as economic factors are also investigated. Finally, using simulated and real-world data examples, it is shown that for one year ahead, the mean absolute percentage error (MAPE) for the load peak is less than 5%. Moreover, for the 24-h pattern forecasting, the mean of MAPE for all days is less than 5%.

show abstract

“…The forecasting method can be the way to solve the problem. Forecasting is a process of predicting based on historical data and extracting trends that can be approached using statistical or machine learning [2]. In [3]they study the GPU NVIDIA GTX 1060, which is affected by the bitcoin price.…”

Section: Introductionmentioning

confidence: 99%

Forecasting of GPU Prices Using Transformer Method

2023

View full text Add to dashboard Cite

GPU or VGA (graphic processing unit) is a vital component of computers and laptops, used for tasks such as rendering videos, creating game environments, and compiling large amounts of code. The price of GPU/VGA has fluctuated significantly since the start of the COVID-19 pandemic in 2020, due in part to the increased demand for GPUs for remote work and online activities. Furthermore, accurate GPU price forecasting can have broader implications beyond the computer hardware industry, with potential applications in investment decision-making, production planning, and pricing strategies for manufacturers. This research aims to forecast future GPU prices using deep learning-based time series forecasting using the Transformer model. We use daily prices of NVIDIA RTX 3090 Founder Edition as a test case. We use historical GPU prices to forecast 8, 16, and 30 days. Moreover, Transformer we compare the results of the Transformer model with two other models, RNN and LSTM. We found that to forecast 30 days; the Transformer model gets a higher coefficient of correlation (CC) of 0.8743, a lower root mean squared error (RMSE) value of 34.68, and a lower mean absolute percentage error (MAPE) of 0.82 compared to the RNN and LSTM model. These results suggest that the model is an effective and efficient method for predicting GPU prices.

show abstract

Short-Term Load Forecasting Based on the Transformer Model

Cited by 35 publications

References 31 publications

Transformer-Based Model for Electrical Load Forecasting

Transformer-Based Model for Electrical Load Forecasting

A New Deep Learning Restricted Boltzmann Machine for Energy Consumption Forecasting

Forecasting of GPU Prices Using Transformer Method

Contact Info

Product

Resources

About