Entropic Trace Estimates for Log Determinants

Fitzsimons, Jack K.; Granziol, Diego; Cutajar, Kurt; Osborne, Michael P.; Filippone, Maurizio; Roberts, Stephen

doi:10.48550/arxiv.1704.07223

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We note that there exist methods to estimate log-determinants of matrices based on power series expansions (e.g. Han et al 2015;Fitzsimons et al 2017;Granziol et al 2018). However, these methods rely on stochastic trace estimations using random probing vectors.…”

Section: Log-determinant Approximationmentioning

confidence: 99%

A novel approach to visibility-space modelling of interferometric gravitational lens observations at high angular resolution

Powell,

Vegetti,

McKean

et al. 2020

Preprint

View full text Add to dashboard Cite

We present a new gravitational lens modelling technique designed to model highresolution interferometric observations with large numbers of visibilities without the need to pre-average the data in time or frequency. We demonstrate the accuracy of the method using validation tests on mock observations. Using small data sets with ∼ 10 3 visibilities, we first compare our approach with the more traditional direct Fourier transform (DFT) implementation and direct linear solver. Our tests indicate that our source inversion is indistinguishable from that of the DFT. Our method also infers lens parameters to within 1 to 2 per cent of both the ground truth and DFT, given sufficiently high signal-to-noise ratio (SNR). When the SNR is as low as 5, both approaches lead to errors of several tens of per cent in the lens parameters and a severely disrupted source structure, indicating that this is an issue related to the data quality rather than the modelling technique of choice. We then analyze a large data set with ∼ 10 8 visibilities and a SNR matching real global Very Long Baseline Interferometry observations of the gravitational lens system MG J0751+2716. The size of the data is such that it cannot be modelled with traditional implementations. Using our novel technique, we find that we can infer the lens parameters and the source brightness distribution, respectively, with an RMS error of 0.25 and 0.97 per cent relative to the ground truth.

show abstract

Section: Log-determinant Approximationmentioning

confidence: 99%

A novel approach to visibility-space modelling of interferometric gravitational lens observations at high angular resolution

Powell,

Vegetti,

McKean

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…For the problem of log determinant, this signifies that the entropy of the spectral approximation should decrease with the addition of every moment constraint. We implement the MaxEnt algorithm proposed in Bandyopadhyay et al [2], which we refer to as OMxnt, in the same manner as applied for log determinant estimation in Fitzsimons et al [30], and compare it against the proposed MEMe approach. Specifically, we show results on the Ecology dataset [31], with n = 999, 999, for which the true log determinant can be calculated.…”

Section: Log Determinant Estimation On Real Datamentioning

confidence: 99%

MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning

Granziol

Zohren

et al. 2019

Entropy

Self Cite

View full text Add to dashboard Cite

Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation. maximum entropy spectral density, which provides an analytic upper bound. Furthermore, in developing our algorithm, we establish equivalence between maximum entropy methods and constrained Bayesian variational inference [10].The main contributions of the paper are as follows:• We propose a maximum entropy algorithm, which is stable and consistent for hundreds of moments, surpassing other off-the-shelf algorithms with a limit of a small number of moments. Based on this robust algorithm, we develop a new Maximum Entropy Method (MEMe) which improves upon the scalability of existing machine learning algorithms by efficiently approximating computational bottlenecks using maximum entropy and fast moment estimation techniques; • We establish the link between maximum entropy methods and variational inference under moment constraints, hence connecting the former to well-known Bayesian approximation techniques; • We apply MEMe to the problem of estimating the log determinant, crucial to inference in determinental point processes [11], and to that of estimating the entropy of a Gaussian mixture, important to state-of-the-art information-theoretic Bayesian optimisation algorithms.

show abstract

Entropic Trace Estimates for Log Determinants

Cited by 2 publications

References 0 publications

A novel approach to visibility-space modelling of interferometric gravitational lens observations at high angular resolution

A novel approach to visibility-space modelling of interferometric gravitational lens observations at high angular resolution

MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning

Contact Info

Product

Resources

About