An ADMM Algorithm for a Class of Total Variation Regularized Estimation Problems

Wahlberg, Bo; Boyd, Stephen; Annergren, Mariette; Wang, Yan

doi:10.3182/20120711-3-be-2027.00310

Cited by 201 publications

(129 citation statements)

References 15 publications

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…Although known by some statisticians, this method seems to be largely ignored, at least in the signal processing community. Evidence of this is that iterative methods are regularly proposed for 1D TV denoising [32]- [34]. To understand the principle of the taut string method, define the sequence of running sums r by r …”

Section: Introductionmentioning

confidence: 99%

A Direct Algorithm for 1-D Total Variation Denoising

Condat

2013

IEEE Signal Process. Lett.

302

280

View full text Add to dashboard Cite

Abstract-A very fast noniterative algorithm is proposed for denoising or smoothing one-dimensional discrete signals, by solving the total variation regularized least-squares problem or the related fused lasso problem. A C code implementation is available on the web page of the author.Index Terms-Total variation, denoising, nonlinear smoothing, fused lasso, regularized least-squares, nonparametric regression, convex nonsmooth optimization, taut string I. INTRODUCTIONThe problem of smoothing a signal, to remove or at least attenuate the noise it contains, has numerous applications in communications, control, machine learning, and many other fields of engineering and science [1]. In this paper, we focus on the numerical implementation of total variation (TV) denoising for one-dimensional (1D) discrete signals; that is, we are given a (noisy) signalN of size N ≥ 1, and we want to efficiently compute the denoised signal x ⋆ ∈ R N , defined implicitly as the solution to the minimization problemfor some regularization parameter λ ≥ 0 (whose choice is a difficult problem by itself [2]). We recall that, as the functional to minimize is strongly convex, the solution x ⋆ to the problem exists and is unique, whatever the data y. The TV denoising problem has received large attention in the communities of signal and image processing, inverse problems, sparse sampling, statistical regression analysis, optimization theory, among others. It is not the purpose of this paper to review the properties of the nonlinear TV denoising filter, since numerous papers can be found on this vast topic; see, e.g., [3]-[6] for various insights. A more general problem, which encompasses TV denoising as a particular case, is the fused lasso signal approximator, introduced in [7], which yields a solution that has sparsity in both the coefficients and their successive differences. It consists in solving the problem Total variation denoising can be interpreted as pulling the discrete primitive r of the signal y taut in a tube around it. The taut string (blue polyline) interpolates the sequence s ⋆ solution to (4) (blue dots), which after discrete differentiation yields the denoised sequence x ⋆ solution to (1). The proposed algorithm is not based on this interpretation.denoising, since the solution z ⋆ can be obtained by simple soft-thresholding from the solution x ⋆ of (1):It is straightforward to add soft-thresholding steps to the proposed algorithm, for essentially the same computation time. So, for simplicity of the exposition, we focus on the TV denoising problem (1) direct, noniterative, way, possibly in-place. It is appropriate for real-time processing of an incoming stream of data, as it locates the jumps in x ⋆ one after the other by forward scans, almost online. The possibility of such an algorithm sheds light on the relatively local nature of the TV denoising filter [25].After this work was completed, the author found that there already exists a direct, linear time, method for 1D TV denoising, called the taut string algorithm [26], see al...

show abstract

Section: Introductionmentioning

confidence: 99%

A Direct Algorithm for 1-D Total Variation Denoising

Condat

2013

IEEE Signal Process. Lett.

302

280

View full text Add to dashboard Cite

show abstract

“…Our approach is closely related to a technique called total-variation denoising from the image-processing literature (Rudin et al, 1992), which is also known in statistics as the fused lasso (Tibshirani et al, 2005). We draw heavily on recent work about computationally efficient estimation for this class of optimization problems, including Tibshirani and Taylor (2011), Ramdas and Tibshirani (2014), Wahlberg et al (2012), Tansey et al (2014), Wang et al (2014), and Tansey and Scott (2015). Specifically, we use the algorithm from Tansey and Scott (2015) (which is itself strongly motivated by the discussion in Wang et al, 2014) to solve a series of optimization problems that combine a binomial likelihood together with a total-variation penalty over the nodes of an undirected graph.…”

Section: Statistical Backgroundmentioning

confidence: 99%

Multiscale Spatial Density Smoothing: An Application to Large-Scale Radiological Survey and Anomaly Detection

Tansey

Athey

Reinhart

et al. 2017

Journal of the American Statistical Association

View full text Add to dashboard Cite

We consider the problem of estimating a spatially varying density function, motivated by problems that arise in large-scale radiological survey and anomaly detection. In this context, the density functions to be estimated are the background gamma-ray energy spectra at sites spread across a large geographical area, such as nuclear production and waste-storage sites, military bases, medical facilities, university campuses, or the downtown of a city. Several challenges combine to make this a difficult problem. First, the spectral density at any given spatial location may have both smooth and nonsmooth features. Second, the spatial correlation in these density functions is neither stationary nor locally isotropic. Finally, at some spatial locations, there is very little data. We present a method called multiscale spatial density smoothing that successfully addresses these challenges. The method is based on recursive dyadic partition of the sample space, and therefore shares much in common with other multiscale methods, such as wavelets and Pólya-tree priors. We describe an efficient algorithm for finding a maximum a posteriori (MAP) estimate that leverages recent advances in convex optimization for non-smooth functions.We apply multiscale spatial density smoothing to real data collected on the background gamma-ray spectra at locations across a large university campus. The method exhibits state-of-the-art performance for spatial smoothing in density estimation, and it leads to substantial improvements in power when used in conjunction with existing methods for detecting the kinds of radiological anomalies that may have important consequences for public health and safety.Key words: radiological survey, density estimation, spatial statistics, Bayesian nonparametrics, total-variation denoising, fused lasso IntroductionLost or stolen radioactive sources present a challenging security problem. Widely used for industrial radiography, sterilization, and medical imaging, these sources are often poorly secured (Gaffigan, 2012) and sometimes stolen (Korshukin and Emery, 2006). To prevent dangerous accidents and to detect radiological dispersal devices (dirty bombs) before they can be used, security agencies are interested in continuously monitoring wide areas for radiation sources. A simple method is to monitor overall radiation levels. But these vary naturally in space, as different soil, stone, and building materials can contain widely different amounts of naturally occurring radioactive materials (NORM). Moreover, different detectors exhibit very different overall sensitivities to radiation. It is therefore more effective to monitor the energy spectrum of the detected radiation instead, as different radioactive materials emit gamma radiation at distinct energies.But to find a spectral anomaly, one must first know what the normal background spectrum looks like at all spatial locations. The background spectrum is the probability distribution for the energy of a gamma ray emitted by natural sources of radiation at a g...

show abstract

“…Standard software for convex optimization can be used to solve (1). However, it is possible to use the special structure to derive more efficient optimization algorithms for (1), see [2], [14].…”

Section: Problem Statementmentioning

confidence: 99%

“…Here we have used that the sign function defined by (14) only depends on the sign of its argument and (14) implies that the sign is not changed by the transformation σ 2 t = −1/(2η t ).…”

Section: Proof: First Notice Thatmentioning

confidence: 99%

See 1 more Smart Citation

On l<inf>1</inf> mean and variance filtering

Wahlberg

Rojas

Annergren

2011

2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR)

Self Cite

View full text Add to dashboard Cite

Abstract-This paper addresses the problem of segmenting a time-series with respect to changes in the mean value or in the variance. The first case is when the time data is modeled as a sequence of independent and normal distributed random variables with unknown, possibly changing, mean value but fixed variance. The main assumption is that the mean value is piecewise constant in time, and the task is to estimate the change times and the mean values within the segments. The second case is when the mean value is constant, but the variance can change. The assumption is that the variance is piecewise constant in time, and we want to estimate change times and the variance values within the segments. To find solutions to these problems, we will study an l 1 regularized maximum likelihood method, related to the fused lasso method and l 1 trend filtering, where the parameters to be estimated are free to vary at each sample. To penalize variations in the estimated parameters, the l 1 -norm of the time difference of the parameters is used as a regularization term. This idea is closely related to total variation denoising. The main contribution is that a convex formulation of this variance estimation problem, where the parametrization is based on the inverse of the variance, can be formulated as a certain l 1 mean estimation problem. This implies that results and methods for mean estimation can be applied to the challenging problem of variance segmentation/estimation.

show abstract

An ADMM Algorithm for a Class of Total Variation Regularized Estimation Problems

Cited by 201 publications

References 15 publications

A Direct Algorithm for 1-D Total Variation Denoising

A Direct Algorithm for 1-D Total Variation Denoising

Multiscale Spatial Density Smoothing: An Application to Large-Scale Radiological Survey and Anomaly Detection

On l<inf>1</inf> mean and variance filtering

Contact Info

Product

Resources

About