On sparsity by NUV-EM, Gaussian message passing, and Kalman smoothing

Abstract

Normal priors with unknown variance (NUV) have long been known to promote sparsity and to blend well with parameter learning by expectation maximization (EM). In this paper, we advocate this approach for linear state space models for applications such as the estimation of impulsive signals, the detection of localized events, smoothing with occasional jumps in the state space, and the detection and removal of outliers.The actual computations boil down to multivariate-Gaussian message passing algorithms that are closely related to Kalman smoothing. We give improved tables of Gaussian-message computations from which such algorithms are easily synthesized, and we point out two preferred such algorithms.

Unsupervised feature extraction, signal labeling, and blind signal separation in a state space world

“…Several authors [23]- [25], [27] pointed out that the local maxima of the likelihood function are such that the number of non-zero elements among…”

“…1, all quantities required for EM can be efficiently computed. The modified Bryson-Frazier smoother [25] is a suitable choice of message passing updates for our purposes. In particular, it is stable while any input (co-)variance goes to zero, which is actually expected in our algorithm.…”

“…We here recall the modified Bryson-Frazier smoother [25] and provide the link to the required expectation quantities for EM. The smoother begins with a forward pass for k = 1, .…”

“…Indeed, LSSMs are versatile enough to generate, within a common representation, smoothly-varying signals, such as a wandering baseline, with white Gaussian noise inputs [21], [22] and repetitive signal shapes with sparse inputs, modeled as zero-mean normal variables with unknown variances (NUV) [23]- [25] (cf. Sec.…”

“…III). In addition, the use of the modified Bryson-Frazier smoother [25], [26], which is numerically stable and avoids matrix inversions, allows to compute all expectations required in EM with low complexity (cf. Sec.…”