Space-alternating generalized expectation-maximization algorithm

Abstract

The expectation-maximization (EM) method can facilitate maximizing likelihood functions that arise in statistical estimation problems. In the classical EM paradigm, one iteratively maximizes the conditional log-likelihood of a single unobservable complete data space, rather than maximizing the intractable likelihood function for the measured or incomplete data. EM algorithms update all parameters simultaneously, which has two drawbacks: 1) slow convergence, and 2) difficult maximization steps due to coupling when smoothness penalties are used. This paper describes the space-alternating generalized EM (SAGE) method, which updates the parameters sequentially by alternating between several small hidden-data spaces defined by the algorithm designer. We prove that the sequence of estimates monotonically increases the penalized-likelihood objective, we derive asymptotic convergence rates, and we provide sufficient conditions for monotone convergence in norm. Two signal processing applications illustrate the method: estimation of superimposed signals in Gaussian noise, and image reconstruction from Poisson measurements. In both applications, our SAGE algorithms easily accommodate smoothness penalties and converge faster than the EM algorithms.

Spatiotemporal multi-resolution approximation of the Amari type neural field model

“…One solution to this problem could be expectation-conditional maximization (ECM) (Meng and Rubin, 1993;Meng, 1994), which replaces the M-step by a series of computationally simplified conditional maximization (CM) steps. The ECM is a class of generalized EM (GEM) algorithms in which the Q-function is increased rather than being maximized (Fessler and Hero, 1994). For systems with a high number of states, the ensemble Kalman smoother (EnKS) (Evensen, 2003(Evensen, , 2009b provides an alternative to the RTS smoother used in this work.…”

Equalization of Time-Varying Channels

“…So called generalized EM algorithms, which increase but do not necessarily maximally increase the likelihood at each iteration, have also been proposed (e.g., [FH94]). The EM recursion is sometimes regarded as having two separate steps: an "E step" which computes the conditional expectation (i.e., the integral) in (6.91), and an "M step" which performs the maximization in (6.91).…”

Multipath channel identification by using global optimization in ambiguity function domain

“…One of the most popular approach to obtain more efficient ML estimates is the EM algorithm [2]. To further improve the speed of convergence of the EM approach, SAGE algorithm has been proposed [3]. In SAGE, parameters are updated sequentially in lower dimensional parameter spaces.…”

“…Various different forms of the EM algorithm have been developed to further improve the performance. The most popular one is the space alternating generalized EM (SAGE) algorithm, which was developed by Fessler and Hero [3]. In SAGE, parameters are updated sequentially in contrast with the EM where all the parameters are updated simultaneously.…”