Variational Inference for Large-Scale Models of Discrete Choice

Abstract

Discrete choice models are commonly used by applied statisticians in numerous fields, such as marketing, economics, finance, and operations research. When agents in discrete choice models are assumed to have differing preferences, exact inference is often intractable. Markov chain Monte Carlo techniques make approximate inference possible, but the computational cost is prohibitive on the large data sets now becoming routinely available. Variational methods provide a deterministic alternative for approximation of the posterior distribution. We derive variational procedures for empirical Bayes and fully Bayesian inference in the mixed multinomial logit model of discrete choice. The algorithms require only that we solve a sequence of unconstrained optimization problems, which are shown to be convex. Extensive simulations demonstrate that variational methods achieve accuracy competitive with Markov chain Monte Carlo, at a small fraction of the computational cost. Thus, variational methods permit inferences on data sets that otherwise could not be analyzed without bias-inducing modifications to the underlying model.

Personalized market response analysis for a wide variety of products from sparse transaction data

“…Our model is similar to adaptive personalization systems proposed by [3,8,10,31]. However, it differs in that our model structure facilitates analysis of widely various product categories.…”

Exploring Time-Sensitive Variational Bayesian Inference LDA for Social Media Data

“…The VB approach [6,12] approximates the variational distribution by minimising the distance from the true distribution. Specifically, an expectation maximization (EM) algorithm is used to maximise the lower bound of the loglikelihood of all documents, which equivalently minimises the distance between the variational distribution and the true posterior distribution.…”

“…Importantly, since the VB approach does not have the topic assignment step, the single topic assignment strategy (mentioned in the introduction and discussed further in Section 3) cannot be applied. The main advantage of the VB approach is that the lower bound converges much more quickly than the sampling approach especially on large datasets [6,12]. Moreover, the VB approach can be intuitively implemented in parallel since the updates of γ d & φ d among documents do not impact each other, while the sampling approach cannot be easily parallelised as it is intrinsically sequential [15].…”

“…For example, Hoffman et al [20] and Braun et al [12] recently proposed a VB-based solution to quickly inference a large number of documents. In this paper, we offer a solution to apply an enhanced VB approach (TVB) for social media data, which incorporates time in the topic modelling process.…”

“…However, another LDA implementation, the Variational Bayesian (VB)-based topic modelling approach, has not been well studied on social media posts. As the VB approach has been shown to be more efficient for large datasets [6,12], it can be argued that VB can better handle the increasing volume and dynamicity of social media data.…”