Rényi divergence based covariance matching pursuit of joint sparse support

Khanna, Saurabh; Murthy, Chandra R.

doi:10.1109/spawc.2017.8227671

Cited by 7 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have provided a unifying theoretical platform for deriving different sparse Bayesian learning algorithms for electromagnetic brain imaging using the Majorization-Minimization (MM) framework. [86], Rényi [87], Itakura-Saito (IS) [88], [89] and β divergences [90]- [94] as well as transportation metrics such as the Wasserstein distance between empirical and statistical covariances (e.g., [95]- [98]).…”

Section: Discussionmentioning

confidence: 99%

“…It is conceivable that alternative divergence metrics can be used for solving the M/EEG source reconstruction problem in the future by modeling specific neurophysiologically valid aspects of similarity between data and model output. Promising metrics in that respect are information divergences such as Kullback-Leibler (KL) [92], Rényi [90], Itakura-Saito (IS) [93] and β divergences [94]- [97] as well as transportation metrics such as the Wasserstein distance between empirical and statistical covariances (e.g., [98]- [101]).…”

Section: Discussionmentioning

confidence: 99%

“…Type-I approaches measure first-order differences between modeled and reconstructed time series using variants of the MSE, while Type-II approaches amount to using the log-det Bregman divergence to measure differences in the second-order statistics of the empirically observed and modeled data as summarized in the respective covariance matrices. While the connection between the Type-II loss function and the log-det Bregman divergence has been investigated and exploited in numerous forms such as Stein's loss [50] or the graphical Lasso [77], [84], [85], and has found applications in disciplines such as information theory and metric learning [86], [87], wireless communication [21], and signal processing [26], [88], [89], it has not received much attention in the May 17, 2021 DRAFT BSI literature to the best of authors' knowledge. Here, we have used this insight to devise a novel cross-validation scheme, temporal CV, in which model fit is measured in terms of the log-det Bregman divergence (or, Type-II likelihood) on held-out samples.…”

Section: B Interpretation Of Type-i and Type-ii Loss Functions As Divergencesmentioning

confidence: 99%

See 2 more Smart Citations

Unification of Sparse Bayesian Learning Algorithms for Electromagnetic Brain Imaging with the Majorization Minimization Framework

Hashemi

Cai

Kutyniok

et al. 2020

Preprint

View full text Add to dashboard Cite

Methods for electro- or magnetoencephalography (EEG/MEG) based brain source imaging (BSI) using sparse Bayesian learning (SBL) have been demonstrated to achieve excellent performance in situations with low numbers of distinct active sources, such as event-related designs. This paper extends the theory and practice of SBL in three important ways. First, we reformulate three existing SBL algorithms under the majorization-minimization (MM) framework. This unification perspective not only provides a useful theoretical framework for comparing different algorithms in terms of their convergence behavior, but also provides a principled recipe for constructing novel algorithms with specific properties by designing appropriate bounds of the Bayesian marginal likelihood function. Second, building on the MM principle, we propose a novel method called LowSNR-BSI that achieves favorable source reconstruction performance in low signal-to-noise-ratio settings. Third, precise knowledge of the noise level is a crucial requirement for accurate source reconstruction. Here we present a novel principled technique to accurately learn the noise variance from the data either jointly within the source reconstruction procedure or using one of two proposed cross-validation strategies. Empirically, we could show that the monotonous convergence behavior predicted from MM theory is confirmed in numerical experiments. Using simulations, we further demonstrate the advantage of LowSNR-BSI over conventional SBL in low-SNR regimes, and the advantage of learned noise levels over estimates derived from baseline data. To demonstrate the usefulness of our novel approach we show neurophysiologically plausible source reconstructions on averaged auditory evoked potential data.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: B Interpretation Of Type-i and Type-ii Loss Functions As Divergencesmentioning

confidence: 99%

See 1 more Smart Citation

Unification of Sparse Bayesian Learning Algorithms for Electromagnetic Brain Imaging with the Majorization Minimization Framework

Hashemi

Cai

Kutyniok

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…While the use of column 3 While the m×n matrices A and B may represent highly underdetermined linear systems (when m n), their m 2 ×n sized Khatri-Rao product A B can become an overdetermined system. In fact, many covariance matching based sparse support recovery algorithms [16], [17], [29] exploit this fact to offer significantly better support reconstruction performance. normalized sensing matrices is commonplace in compressive sensing, we are often interested in showing the restricted isometry of the Khatri-Rao product of randomly constructed input matrices with columns being normalized only in the average sense.…”

Section: Probabilistic K-ric Boundmentioning

confidence: 99%

On the Restricted Isometry of the Columnwise Khatri–Rao Product

Khanna

Murthy

2018

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

The columnwise Khatri-Rao product of two matrices is an important matrix type, reprising its role as a structured sensing matrix in many fundamental linear inverse problems. Robust signal recovery in such inverse problems is often contingent on proving the restricted isometry property (RIP) of a certain system matrix expressible as a Khatri-Rao product of two matrices. In this work, we analyze the RIP of a generic columnwise Khatri-Rao product matrix by deriving two upper bounds for its k th order Restricted Isometry Constant (k-RIC) for different values of k. The first RIC bound is computed in terms of the individual RICs of the input matrices participating in the Khatri-Rao product. The second RIC bound is probabilistic, and is specified in terms of the input matrix dimensions. We show that the Khatri-Rao product of a pair of m×n sized random matrices comprising independent and identically distributed subgaussian entries satisfies k-RIP with arbitrarily high probability, provided m exceeds O(k log n). Our RIC bounds confirm that the Khatri-Rao product exhibits stronger restricted isometry compared to its constituent matrices for the same RIP order. The proposed RIC bounds are potentially useful in the sample complexity analysis of several sparse recovery problems.

show abstract

“…Also, several algorithms from the single sample setting have been generalized to work with multiple samples that include convex programming methods [20], [28], [11], thresholding-based methods [13], [14], Bayesian methods [34] and greedy methods [29], [30]. However, none of the above works addresses the question of tradeoff between m and n when m < k. Initial works considering the m < k regime were [21] and [4], followed by [18] and [23], where it was empirically demonstrated that when multiple samples are available, it is possible to operate in the m < k regime. However, the analysis in [4] is done under two fairly restrictive conditions.…”

Section: Introductionmentioning

confidence: 99%

Sample-Measurement Tradeoff in Support Recovery Under a Subgaussian Prior

Ramesh

Murthy

Tyagi

2019

2019 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

Data samples from Ê d with a common support of size k are accessed through m random linear projections per sample. It is well-known that roughly k measurements from a single sample are sufficient to recover the support. In the multiple sample setting, do k overall measurements still suffice when only m < k measurements per sample are allowed? We answer this question in the negative by considering a generative model setting with independent samples drawn from a subgaussian prior. We show that n = Θ((k 2 /m 2 ) · log k(d − k)) samples are necessary and sufficient to recover the support exactly. In turn, this shows that k overall samples are insufficient for support recovery when m < k; instead we need about k 2 /m overall measurements. Our proposed sample-optimal estimator has a closed-form expression, has computational complexity of O(dnm), and is of independent interest.

show abstract

Rényi divergence based covariance matching pursuit of joint sparse support

Cited by 7 publications

References 19 publications

Unification of Sparse Bayesian Learning Algorithms for Electromagnetic Brain Imaging with the Majorization Minimization Framework

Unification of Sparse Bayesian Learning Algorithms for Electromagnetic Brain Imaging with the Majorization Minimization Framework

On the Restricted Isometry of the Columnwise Khatri–Rao Product

Sample-Measurement Tradeoff in Support Recovery Under a Subgaussian Prior

Contact Info

Product

Resources

About