Effective Snapshot Compressive-spectral Imaging via Deep Denoising and Total Variation Priors

Qiu, Haiquan; Wang, Yao; Meng, Deyu

doi:10.1109/cvpr46437.2021.00901

Cited by 28 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To further improve the model, we can introduce some regularization terms 9,16 or priors, 37,38 which are good at denoising. The proposed gradient fusion model is an iterative algorithm to fuses three priors, so that it fails to achieve a greater advantage in imaging speed compared with Refine-GAN and CDDN+TDC.…”

Section: Discussionmentioning

confidence: 99%

Deep compressed sensing MRI via a gradient‐enhanced fusion model

Dai

Wang

2023

Medical Physics

View full text Add to dashboard Cite

Background: Compressed sensing has been employed to accelerate magnetic resonance imaging by sampling fewer measurements. However, conventional iterative optimization-based CS-MRI are time-consuming for iterative calculations and often share poor generalization ability on multicontrast datasets. Most deep-learning-based CS-MRI focus on learning an end-to-end mapping while ignoring some prior knowledge existed in MR images. Purpose: We propose an iterative fusion model to integrate the image and gradient-based priors into reconstruction via convolutional neural network models while maintaining high quality and preserving the detailed information as well. Methods: We propose a gradient-enhanced fusion network (GFN) for fast and accurate MRI reconstruction, in which dense blocks with dilated convolution and dense residual learnings are used to capture abundant features with fewer parameters. Meanwhile, decomposed gradient maps containing obvious structural information are introduced into the network to enhance the reconstructed images. Besides this, gradient-based priors along directions X and Y are exploited to learn adaptive tight frames for reconstructing the desired image contrast and edges by respective gradient fusion networks. After that, both image and gradient priors are fused in the proposed optimization model, in which we employ the l 2 -norm to promote the sparsity of gradient priors. The proposed fusion model can effectively help to capture edge structures in the gradient images and to preserve more detailed information of MR images. Results: Experimental results demonstrate that the proposed method outperforms several CS-MRI methods in terms of peak signal-to-noise (PSNR), the structural similarity index (SSIM), and visualizations on three sampling masks with different rates. Noteworthy, to evaluate the generalization ability, the proposed model conducts cross-center training and testing experiments for all three modalities and shares more stable performance compared than other approaches. In addition, the proposed fusion model is applied to other comparable deep learning methods.The quantitative results show that the reconstruction results of these methods are obviously improved. Conclusions: The gradient-based priors reconstructed from GFNs can effectively enhance the edges and details of under-sampled data. The proposed fusion model integrates image and gradient priors using l 2 -norm can effectively improve the generalization ability on multicontrast datasets reconstruction.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep compressed sensing MRI via a gradient‐enhanced fusion model

Dai

Wang

2023

Medical Physics

View full text Add to dashboard Cite

show abstract

“…Apart from ADMM, other variable splitting algorithms, such as half quadratic splitting [18], have been adopted in the PnP paradigms [16,19]. More recent PnP methods achieved promising IR results with the help of state-of-the-art deep image denoisers [12,13,15]. For example, Qiu et al [20] combined a classical denoiser (TV [21]) and a deep denoiser (FFDNet [22]) in PnP to boost the quality of reconstructed images.…”

Section: Plug-and-play For Image Restorationmentioning

confidence: 99%

“…More recent PnP methods achieved promising IR results with the help of state-of-the-art deep image denoisers [12,13,15]. For example, Qiu et al [20] combined a classical denoiser (TV [21]) and a deep denoiser (FFDNet [22]) in PnP to boost the quality of reconstructed images. Besides, several theoretical analyses on convergence of PnP methods to a fixed point have been investigated [14].…”

Section: Plug-and-play For Image Restorationmentioning

confidence: 99%

“…For example, the LISTA scheme [7] proposed to unfold the iterative shrinkage thresholding algorithm (ISTA) [8] for sparse coding, which demonstrated superior results in IR tasks such as super-resolution [9]. More recently, the Plug-and-Play (PnP) scheme [10] has gained great popularity due to its effectiveness and flexibility for solving a wide range of inverse problems, e.g., medical imaging [11], snapshot compressive imaging [12,13] and image restoration [14][15][16]. Different from many unrolling methods, PnP directly adopts an off-the-shelf deep denoiser to replace the proximal mapping (involving the regularizer) inside an iterative reconstruction algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

REPNP: Plug-and-Play with Deep Reinforcement Learning Prior for Robust Image Restoration

Wang¹,

Zhang²,

Ravishankar³

et al. 2022

Preprint

View full text Add to dashboard Cite

Image restoration schemes based on the pre-trained deep models have received great attention due to their unique flexibility for solving various inverse problems. In particular, the Plug-and-Play (PnP) framework is a popular and powerful tool that can integrate an offthe-shelf deep denoiser for different image restoration tasks with known observation models. However, obtaining the observation model that exactly matches the actual one can be challenging in practice. Thus, the PnP schemes with conventional deep denoisers may fail to generate satisfying results in some real-world image restoration tasks. We argue that the robustness of the PnP framework is largely limited by using the off-the-shelf deep denoisers that are trained by deterministic optimization. To this end, we propose a novel deep reinforcement learning (DRL) based PnP framework, dubbed RePNP, by leveraging a light-weight DRL-based denoiser for robust image restoration tasks. Experimental results demonstrate that the proposed RePNP is robust to the observation model used in the PnP scheme deviating from the actual one. Thus, RePNP can generate more reliable restoration results for image deblurring and super resolution tasks. Compared with several state-of-the-art deep image restoration baselines, RePNP achieves better results subjective to model deviation with fewer model parameters.

show abstract

“…However, due to limited priors with the training sets, Deep denoiser networks are required to extract artifacts in the reconstruction process, leading to confusing results. To take advantage of both the Deep denoiser network and TV model, H. Qiu et al proposed a combined denoiser TV+FFDNet and achieved superior performance to previous algorithms [18].…”

Section: Introductionmentioning

confidence: 99%

Compressed fluorescence lifetime imaging via combined TV-based and deep priors

Wang

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

Compressed fluorescence lifetime imaging (Compressed-FLIM) is a novel Snapshot compressive imaging (SCI) method for single-shot widefield FLIM. This approach has the advantages of high temporal resolution and deep frame sequences, allowing for the analysis of FLIM signals that follow complex decay models. However, the precision of Compressed-FLIM is limited by reconstruction algorithms. To improve the reconstruction accuracy of Compressed-FLIM in dealing with large-scale FLIM problem, we developed a more effective combined prior model 3DTGp V_net, based on the Plug and Play (PnP) framework. Extensive numerical simulations indicate the proposed method eliminates reconstruction artifacts caused by the Deep denoiser networks. Moreover, it improves the reconstructed accuracy by around 4dB (peak signal-to-noise ratio; PSNR) over the state-of-the-art TV+FFDNet in test data sets. We conducted the single-shot FLIM experiment with different Rhodamine reagents and the results show that in practice, the proposed algorithm has promising reconstruction performance and more negligible lifetime bias.

show abstract

Effective Snapshot Compressive-spectral Imaging via Deep Denoising and Total Variation Priors

Cited by 28 publications

References 29 publications

Deep compressed sensing MRI via a gradient‐enhanced fusion model

Deep compressed sensing MRI via a gradient‐enhanced fusion model

REPNP: Plug-and-Play with Deep Reinforcement Learning Prior for Robust Image Restoration

Compressed fluorescence lifetime imaging via combined TV-based and deep priors

Contact Info

Product

Resources

About