Frequentist Consistency of Variational Bayes

Wang, Yixin; Blei, David M.

doi:10.1080/01621459.2018.1473776

Cited by 143 publications

(128 citation statements)

References 78 publications

Supporting

Mentioning

117

Contrasting

Order By: Relevance

“…Because the ELBO is convex with respect to each of the variational factors, the ELBO is guaranteed to converge to a local optimum (Boyd and Vandenberghe, 2004). Moreover, an important result from the frequentist perspective is the variational Bernstein-von Mises theorem, which states that under benign conditions, the mean-field variational Bayes estimateθ = θ q * (θ )dθ is consistent (Wang and Blei, 2018).…”

Section: Figure 1: Schematic Representations Of Markov Chain Monte Camentioning

confidence: 99%

Bayesian estimation of mixed multinomial logit models: Advances and simulation-based evaluations

Bansal

Krueger

Bierlaire

et al. 2020

Transportation Research Part B: Methodological

View full text Add to dashboard Cite

Variational Bayes (VB) methods have emerged as a fast and computationally-efficient alternative to Markov chain Monte Carlo (MCMC) methods for scalable Bayesian estimation of mixed multinomial logit (MMNL) models. It has been established that VB is substantially faster than MCMC at practically no compromises in predictive accuracy. In this paper, we address two critical gaps concerning the usage and understanding of VB for MMNL. First, extant VB methods are limited to utility specifications involving only individual-specific taste parameters. Second, the finite-sample properties of VB estimators and the relative performance of VB, MCMC and maximum simulated likelihood estimation (MSLE) are not known. To address the former, this study extends several VB methods for MMNL to admit utility specifications including both fixed and random utility parameters. To address the latter, we conduct an extensive simulation-based evaluation to benchmark the extended VB methods against MCMC and MSLE in terms of estimation times, parameter recovery and predictive accuracy. The results suggest that all VB variants with the exception of the ones relying on an alternative variational lower bound constructed with the help of the modified Jensen's inequality perform as well as MCMC and MSLE at prediction and parameter recovery. In particular, VB with nonconjugate variational message passing and the delta-method (VB-NCVMP-∆) is up to 16 times faster than MCMC and MSLE. Thus, VB-NCVMP-∆ can be an attractive alternative to MCMC and MSLE for fast, scalable and accurate estimation of MMNL models.

show abstract

Section: Figure 1: Schematic Representations Of Markov Chain Monte Camentioning

confidence: 99%

Bayesian estimation of mixed multinomial logit models: Advances and simulation-based evaluations

Bansal

Krueger

Bierlaire

et al. 2020

Transportation Research Part B: Methodological

View full text Add to dashboard Cite

show abstract

“…This proposition builds on [19,Theorem 5], which shows that modulo Assumptions 3.1, 3.3, and 3.4, the NVB approximate posterior distribution is asymptotically consistent(see Corollary 3.1). Using the consistency of q * (θ X n ) and Assumption 3.2(1), we first establish the pointwise convergence of H q * (a) to H 0 (a).…”

Section: Analysis Of the Nvb Decision Rulementioning

confidence: 79%

“…The primary result in this section shows that the loss-calibrated approximate posterior q * a (θ X n ) for any a ∈ A is consistent and converges to the Dirac-delta distribution at θ 0 . We establish the frequentist consistency of LCVB approximate posterior, extending and building on the results in [19].…”

Section: Consistency Of the Lcvb Approximate Posteriormentioning

confidence: 85%

See 1 more Smart Citation

Asymptotic consistency of loss‐calibrated variational Bayes

Jaiswal

Honnappa

Rao

2020

Stat

View full text Add to dashboard Cite

This paper establishes the asymptotic consistency of the loss-calibrated variational Bayes (LCVB) method. LCVB was proposed in [12] as a method for approximately computing Bayesian posteriors in a 'loss aware' manner. This methodology is also highly relevant in general data-driven decision-making contexts. Here, we not only establish the asymptotic consistency of the calibrated approximate posterior, but also the asymptotic consistency of decision rules. We also establish the asymptotic consistency of decision rules obtained from a 'naive' variational Bayesian procedure.

show abstract

“…This method will converge to a local maximum of the evidence lower bound (Lange, 2013). Wang and Blei (2017) show that under certain conditions, Bayesian point estimators extracted from the variational Bayes solution are consistent. …”

Section: Mean-field Variational Approximationmentioning

confidence: 99%

Frequency Tracking for Speed Estimation

Lindfors¹

2018

View full text Add to dashboard Cite

Estimating the frequency of a periodic signal, or tracking the time-varying frequency of an almost periodic signal, is an important problem that is well studied in literature. This thesis focuses on two subproblems where contributions can be made to the existing theory: frequency tracking methods and measurements containing outliers.Maximum-likelihood-based frequency estimation methods are studied, focusing on methods which can handle outliers in the measurements. Katkovnik's frequency estimation method is generalized to real and harmonic signals, and a new method based on expectation-maximization is proposed. The methods are compared in a simulation study in which the measurements contain outliers. The proposed methods are compared with the standard periodogram method.Recursive Bayesian methods for frequency tracking are studied, focusing on the Rao-Blackwellized point mass filter (rbpmf). Two reformulations of the rbpmf aiming to reduce computational costs are proposed. Furthermore, the technique of variational approximate Rao-Blackwellization is proposed, which allows usage of a Student's t distributed measurement noise model. This enables recursive frequency tracking methods to handle outliers using heavy-tailed noise models in Rao-Blackwellized filters such as the rbpmf. A simulation study illustrates the performance of the methods when outliers occur in the measurement noise.The framework above is applied to and studied in detail in two applications. The first application is on frequency tracking of engine sound. Microphone measurements are used to track the frequency of Doppler-shifted variants of the engine sound of a vehicle moving through an area. These estimates can be used to compute the speed of the vehicle. Periodogram-based methods and the rbpmf are evaluated on simulated and experimental data. The results indicate that the rbpmf has lower rmse than periodogram-based methods when tracking fast changes in the frequency.The second application relates to frequency tracking of wheel vibrations, where a car has been equipped with an accelerometer. The accelerometer measurements are used to track the frequency of the wheel axle vibrations, which relates to the wheel rotational speed. The velocity of the vehicle can then be estimated without any other sensors and without requiring integration of the accelerometer measurements. In situations with high signal-to-noise ratio (snr), the methods perform well. To remedy situations when the methods perform poorly, an accelerometer input is introduced to the formulation. This input is used to predict changes in the frequency for short time intervals.v Populärvetenskaplig sammanfattningPeriodiska signaler förekommer ofta i praktiken. I många tillämpningar är det intressant att försöka skatta frekvensen av dessa periodiska signaler, eller vibrationer, genom mätningar av dem. Detta kallas för frekvensskattning eller frekvensföljning beroende på om frekvensen är konstant eller varierar över tid. Två tillämpningar studeras i denna licentiatavhandling. Målet i...

show abstract

Frequentist Consistency of Variational Bayes

Cited by 143 publications

References 78 publications

Bayesian estimation of mixed multinomial logit models: Advances and simulation-based evaluations

Bayesian estimation of mixed multinomial logit models: Advances and simulation-based evaluations

Asymptotic consistency of loss‐calibrated variational Bayes

Frequency Tracking for Speed Estimation

Contact Info

Product

Resources

About