The Ensemble Method for Censored Demand Prediction

Abstract

Many economic applications including optimal pricing and inventory management requires prediction of demand based on sales data and estimation of sales reaction to a price change. There is a wide range of econometric approaches which are used to correct a bias in estimates of demand parameters on censored sales data. These approaches can also be applied to various classes of machine learning models to reduce the prediction error of sales volume. In this study we construct two ensemble models for demand prediction with and without accounting for demand censorship. Accounting for sales censorship is based on the idea of censored quantile regression method where the model estimation is splitted on two separate parts: a) prediction of zero sales by classification model; and b) prediction of non-zero sales by regression model. Models with and without accounting for censorship are based on the predictions aggregations of Least squares, Ridge and Lasso regressions and Random Forest model. Having estimated the predictive properties of both models, we empirically test the best predictive power of the model that takes into account the censored nature of demand. We also show that machine learning method with censorship accounting provide bias corrected estimates of demand sensitivity for price change similar to econometric models.

Interpretable machine learning for demand modeling with high-dimensional data using Gradient Boosting Machines and Shapley values

“…One of the main problems of sales forecasting is that due to the observational nature of data, we have to assume that the demand is always met and, therefore, sales reflect the true demand. Even though this assumption is rather realistic taking into account that in our study we are considering products with a long shelf life, the quantity demanded can be larger than the number of units sold, especially in the case of perishable products (Ozhegov and Teterina 2018). Therefore, we recommend that retailers account for whether sales reflect censored demand or not whenever possible (at least, by keeping track of whether and how long the SKU was out of stock for some time during each week).…”