Learning Convex Regularizers for Optimal Bayesian Denoising

Nguyen, Ha Q.; Bostan, Emrah; Unser, Michaël

doi:10.1109/tsp.2017.2777407

Cited by 11 publications

(6 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For given noise statistics the regularization parameter could be learned; however, it would perform sub optimally for different noise statistics and would require retraining. Future systems should learn regularization parameters that can be adapted post training to account for variable noise levels similar to in [55].…”

Section: Discussionmentioning

confidence: 99%

Physics-Based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging

Kellman

Bostan

Repina

et al. 2019

IEEE Trans. Comput. Imaging

Self Cite

137

View full text Add to dashboard Cite

Coded illumination can enable quantitative phase microscopy of transparent samples with minimal hardware requirements. Intensity images are captured with different source patterns, then a nonlinear phase retrieval optimization reconstructs the image. The nonlinear nature of the processing makes optimizing the illumination pattern designs complicated. The traditional techniques for the experimental design (e.g., condition number optimization, and spectral analysis) consider only linear measurement formation models and linear reconstructions. Deep neural networks (DNNs) can efficiently represent the nonlinear process and can be optimized over via training in an end-to-end framework. However, DNNs typically require a large amount of training examples and parameters to properly learn the phase retrieval process, without making use of the known physical models. In this paper, we aim to use both our knowledge of the physics and the power of machine learning together. We propose a new data-driven approach for optimizing coded-illumination patterns for an LED array microscope for a given phase reconstruction algorithm. Our method incorporates both the physics of the measurement scheme and the nonlinearity of the reconstruction algorithm into the design problem. This enables efficient parameterization, which allows us to use only a small number of training examples to learn designs that generalize well in the experimental setting without retraining. We show experimental results for both a well-characterized phase target and mouse fibroblast cells, using coded-illumination patterns optimized for a sparsity-based phase reconstruction algorithm. Our learned design results using two measurements demonstrate similar accuracy to Fourier ptychography with 69 measurements.

show abstract

Section: Discussionmentioning

confidence: 99%

Physics-Based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging

Kellman

Bostan

Repina

et al. 2019

IEEE Trans. Comput. Imaging

Self Cite

137

View full text Add to dashboard Cite

show abstract

“…In recent years, researchers have considered more general parametric nonlinearities whose weights are learned during training. Such models involve linear combinations of Gaussian radial-basis functions [28] and cubic B-splines [29], [30].…”

Section: B Link With Iterative Soft-thresholding Algorithmsmentioning

confidence: 99%

Learning Activation Functions in Deep (Spline) Neural Networks

Bohra

Campos

Gupta

et al. 2020

IEEE Open J. Signal Process.

Self Cite

View full text Add to dashboard Cite

We develop an efficient computational solution to train deep neural networks (DNN) with free-form activation functions. To make the problem well-posed, we augment the cost functional of the DNN by adding an appropriate shape regularization: the sum of the second-order total-variations of the trainable nonlinearities. The representer theorem for DNNs tells us that the optimal activation functions are adaptive piecewise-linear splines, which allows us to recast the problem as a parametric optimization. The challenging point is that the corresponding basis functions (ReLUs) are poorly conditioned and that the determination of their number and positioning is also part of the problem. We circumvent the difficulty by using an equivalent B-spline basis to encode the activation functions and by expressing the regularization as an 1-penalty. This results in the specification of parametric activation function modules that can be implemented and optimized efficiently on standard development platforms. We present experimental results that demonstrate the benefit of our approach.

show abstract

“…Recalling the ADMM algorithm from Section 4.4, we know that solving a regularized inverse problem involves applying the proximal operator associated with the potential function. We can implicitly specify the potential function by learning the proximal operator, e.g., by parameterizing it using 1D B-splines [55,56]. We can see prox learning as a generalization of tuning the regularization weight: instead of scaling a known function, we deform a parametric function.…”

Section: Learning the Regularization Termmentioning

confidence: 99%