Shrinkage priors for high-dimensional demand estimation

Abstract

Estimating demand for large assortments of differentiated goods requires the specification of a demand system that is sufficiently flexible. However, flexible models are highly parameterized so estimation requires appropriate forms of regularization to avoid overfitting. In this paper, we study the specification of Bayesian shrinkage priors for pairwise product substitution parameters. We use a log-linear demand system as a leading example. Log-linear models are parameterized by own and cross-price elasticities, and the total number of elasticities grows quadratically in the number of goods. Traditional regularized estimators shrink regression coefficients towards zero which can be at odds with many economic properties of price effects. We propose a hierarchical extension of the class of global-local priors commonly used in regression modeling to allow the direction and rate of shrinkage to depend on a product classification tree. We use both simulated data and retail scanner data to show that, in the absence of a strong signal in the data, estimates of price elasticities and demand predictions can be improved by imposing shrinkage to higher-level group elasticities rather than zero.

Discussion of “Multivariate dynamic modeling for Bayesian forecasting of business revenue” by A. K. Yanchenko et al.

“…Ruiz et al 5 use a word embedding approach whereas Donnelly et al 15 use matrix factorization. More in keeping with the DLM approach of the paper is Smith and Griffin 16 who show how a high-dimensional log-linear model can be estimated using horseshoe-based shrinkage priors at a product rather than category level using a product classification tree. One benefit of shrinkage models in this context is that products whose price does not vary or very rarely varies in the sample can be included rather than removed from the analysis as is commonly done.…”