A new variational Bayesian algorithm with application to human mobility pattern modeling

Abstract

A new variational Bayesian (VB) algorithm, split and eliminate VB (SEVB), for modeling data via a Gaussian mixture model (GMM) is developed. This new algorithm makes use of component splitting in a way that is more appropriate for analyzing a large number of highly heterogeneous spiky spatial patterns with weak prior information than existing VB-based approaches. SEVB is a highly computationally efficient approach to Bayesian inference and like any VB-based algorithm it can perform model selection and parameter value estimation simultaneously. A significant feature of our algorithm is that the fitted number of components is not limited by the initial proposal giving increased modeling flexibility. We introduce two types of split operation in addition to proposing a new goodnessof-fit measure for evaluating mixture models. We evaluate their usefulness through empirical studies. In addition, we illustrate the utility of our new approach in an application on modeling human mobility patterns. This application involves large volumes of highly heterogeneous spiky data; it is difficult to model this type of data well using the standard VB approach as it is too restrictive and lacking in the flexibility required. Empirical results suggest that our algorithm has also improved upon the goodness-of-fit that would have been achieved using the standard VB method, and that it is also more robust to various initialization settings.

Regression Density Estimation With Variational Methods and Stochastic Approximation

“…Blei and Jordan (2006) considered VA for Dirichlet process mixture models. Recently, Wu, McGrory, and Pettitt (2012) considered variational methods for fitting mixtures for data that require models with narrow, widely separated mixture components. They discussed sophisticated split-and-merge algorithms, building on earlier related methods, such as those in Ghahramani and Beal (2000), Ueda and Ghahramani (2002), and Constantinopoulos and Likas (2007), for simultaneous model selection and parameter estimation as well as on novel criteria for model evaluation.…”

Variational Approximation for Mixtures of Linear Mixed Models

“…Ueda and Ghahramani (2002) proposed a variational Bayesian (VB) split and merge EM procedure to optimize an objective function that allows simultaneous estimation of the parameters and number of components in a Gaussian mixture while avoiding local optima. Wu, McGrory, and Pettitt (2012) developed a split and eliminate VB algorithm that attempts to split all poorly fitted components at the same time and made use of the component-elimination property associated with variational approximation so that no merge moves are required. This component-elimination property was noted previously by Attias (1999) and Corduneanu and Bishop (2001).…”

Identifying and Interpreting Subgroups in Health Care Utilization Data with Count Mixture Regression Models

“…[66] The number of selected components might vary, as mixture models with different values of K can provide good representations of the same data. [67] Our choice of K = 20 balances computational burden with the the goal of our analysis, which is to describe the parameters of the mixture components and the corresponding clusters of observations. More than 20 components/clusters would be unwieldy in our applied setting.…”

“…Further research should investigate how variational Bayesian methods could improve speed and how this affects the accuracy of the estimates. Variational Bayesian mixture models have been found to accurately detect the number of mixture components [67,72] and suffer less from identifiability issues. [73] We limited our consideration to mixtures of parametric (zero-inflated) Negative Binomial regression models.…”