Learning Local Regularization for Variational Image Restoration

Prost, Jean; Houdard, Antoine; Almansa, Andrés; Papadakis, Nicolas

doi:10.1007/978-3-030-75549-2_29

Cited by 16 publications

(13 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our method with established methods from the literature. In particular, we compare with Wasserstein Patch Prior (WPP) [3,22], Expected Patch Log Likelihood (EPLL) [71] and Local Adversarial Regularizer (localAR) [50], as they work on patches and are unsupervised as well. Note that we optimize the EPLL GMM prior using a gradient descent optimizer, as half quadratic splitting proposed by the authors of [71] is much more expensive for the superresolution and CT forward operator.…”

Section: Numerical Examplesmentioning

confidence: 99%

“…• Local Adversarial Regularizer. The adversarial regularizer was introduced in [45] and this framework was recently applied for learning patch-based regularizers (localAR) [50].…”

Section: Quality Measures and Comparisonsmentioning

confidence: 99%

“…For our experiments we used the code of [50], but instead of patch size 15 we used the patch size 6 and replaced the fully convolutional discriminator by a discriminator with 2 convolutional layers, followed by 4 fully connected layers. • Expected Patch Log Likelihood.…”

Section: Quality Measures and Comparisonsmentioning

confidence: 99%

“…where D is a data-fidelity term which depends on the noise model and measures how well the reconstruction fits to the observation and R is a regularizer which copes with the ill-posedness and incorporates prior information. Over the last years, learned regularizers like the total deep variation [35,36] or adversarial regularizers [45,50] as well as extensions of plug-and-play methods [18,62,66] with learned denoisers [23,26,53,69] showed promising results, see [7,48] for an overview. However, even if these methods allow an unsupervised reconstruction, their training is often computationally costly and a huge amount of training images is required.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

PatchNR: Learning from Small Data by Patch Normalizing Flow Regularization

Altekrüger¹,

Denker²,

Hagemann³

et al. 2022

Preprint

View full text Add to dashboard Cite

Learning neural networks using only a small amount of data is an important ongoing research topic with tremendous potential for applications. In this paper, we introduce a regularizer for the variational modeling of inverse problems in imaging based on normalizing flows. Our regularizer, called patchNR, involves a normalizing flow learned on patches of very few images. The subsequent reconstruction method is completely unsupervised and the same regularizer can be used for different forward operators acting on the same class of images. By investigating the distribution of patches versus those of the whole image class, we prove that our variational model is indeed a MAP approach. Our model can be generalized to conditional patchNRs, if additional supervised information is available. Numerical examples for low-dose CT, limited-angle CT and superresolution of material images demonstrate that our method provides high quality results among unsupervised methods, but requires only few data.

show abstract

Section: Numerical Examplesmentioning

confidence: 99%

“…• Local Adversarial Regularizer. The adversarial regularizer was introduced in [45] and this framework was recently applied for learning patch-based regularizers (localAR) [50].…”

Section: Quality Measures and Comparisonsmentioning

confidence: 99%

Section: Quality Measures and Comparisonsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

PatchNR: Learning from Small Data by Patch Normalizing Flow Regularization

Altekrüger¹,

Denker²,

Hagemann³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The connections between neural networks and variational methods have become a topic of intensive research. The idea of learning the regulariser in a variational framework has gained considerable traction and brought the performance of variational models to a new level [23,47,52,58,60]. The closely related idea of unrolling [50,69] the steps of a minimising algorithm for a variational energy and learning its parameters has been equally prominent and successful [1,5,8,13,35,38,39].…”

Section: Related Workmentioning

confidence: 99%

Designing rotationally invariant neural networks from PDEs and variational methods

et al. 2022

View full text Add to dashboard Cite

Partial differential equation models and their associated variational energy formulations are often rotationally invariant by design. This ensures that a rotation of the input results in a corresponding rotation of the output, which is desirable in applications such as image analysis. Convolutional neural networks (CNNs) do not share this property, and existing remedies are often complex. The goal of our paper is to investigate how diffusion and variational models achieve rotation invariance and transfer these ideas to neural networks. As a core novelty, we propose activation functions which couple network channels by combining information from several oriented filters. This guarantees rotation invariance within the basic building blocks of the networks while still allowing for directional filtering. The resulting neural architectures are inherently rotationally invariant. With only a few small filters, they can achieve the same invariance as existing techniques which require a fine-grained sampling of orientations. Our findings help to translate diffusion and variational models into mathematically well-founded network architectures and provide novel concepts for model-based CNN design.

show abstract

Learning from small data sets: Patch‐based regularizers in inverse problems for image reconstruction

Piening,

Altekrüger,

Hertrich

et al. 2024

GAMM-Mitteilungen

View full text Add to dashboard Cite

The solution of inverse problems is of fundamental interest in medical and astronomical imaging, geophysics as well as engineering and life sciences. Recent advances were made by using methods from machine learning, in particular deep neural networks. Most of these methods require a huge amount of data and computer capacity to train the networks, which often may not be available. Our paper addresses the issue of learning from small data sets by taking patches of very few images into account. We focus on the combination of model‐based and data‐driven methods by approximating just the image prior, also known as regularizer in the variational model. We review two methodically different approaches, namely optimizing the maximum log‐likelihood of the patch distribution, and penalizing Wasserstein‐like discrepancies of whole empirical patch distributions. From the point of view of Bayesian inverse problems, we show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods. We demonstrate the power of the methods in computed tomography, image super‐resolution, and inpainting. Indeed, the approach provides also high‐quality results in zero‐shot super‐resolution, where only a low‐resolution image is available. The article is accompanied by a GitHub repository containing implementations of all methods as well as data examples so that the reader can get their own insight into the performance.

show abstract

Learning Local Regularization for Variational Image Restoration

Cited by 16 publications

References 20 publications

PatchNR: Learning from Small Data by Patch Normalizing Flow Regularization

PatchNR: Learning from Small Data by Patch Normalizing Flow Regularization

Designing rotationally invariant neural networks from PDEs and variational methods

Learning from small data sets: Patch‐based regularizers in inverse problems for image reconstruction

Contact Info

Product

Resources

About