Time Series Forecasting Model for Solving Cold Start Problem via Temporal Fusion Transformer

Abstract

Time series forecasting is an important tool in many businesses. It can range from efficiently allocating resources for web traffic, predicting patient needs for staffing requirements, to forecasting a company's product sales. A particular use case, known as "cold start" forecasting, involves making predictions for time series that have little or no historical data, like a new product just entering the retail market. The key assumption of cold start forecasting is that products with similar characteristics should have similar time series trajectories. In such scenarios, traditional forecasting models that heavily rely on past observations may face challenges, necessitating the development of innovative approaches that can effectively make predictions in the absence of a substantial historical dataset. In this paper, Temporal fusion transformer neural network architecture was applied for solving cold start time series forecasting task. Modeling of the method was based on the use of a dataset contained in an open repository. After the preprocessing procedures, the dataset has about 370 time series, each of which has different length of series and has one categorical feature. Categorical feature have only 4 types of different values. For model training was performed to search for optimal hyperparameters across such parameters as: number of attention heads, learning rate, dropout percentage and hidden size.Model performed pretty well on this task. For model comparison were chosen metrics: MAE, RMSE, SMAPE. As can be seen from comparison with such popular models as DeepAR and LSTM, the proposed approach demonstrated the smallest forecasting error. Only one downside is that it can have more problems with anomalies in time series than DeepAR. But at the same time still provide interpretability of results.