A Review of the Deep Learning Methods for Medical Images Super Resolution Problems

Li, Y.; Sixou, Bruno; Peyrin, Françoise

doi:10.1016/j.irbm.2020.08.004

Cited by 191 publications

(95 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Just as important, deep learning provides other avenues for research exploration and development. For example, given the relatively lower resolution of the acquisition image, exploration of the effects of deep learning-based super-resolution might prove worthy of application-specific investigation (9) (see, for example, ). Also, with the same network software libraries, high-performing classification networks can be constructed and trained which might yield novel insights regarding image-based characterization of disease.…”

Section: Discussionmentioning

confidence: 99%

Image- vs. histogram-based considerations in semantic segmentation of pulmonary hyperpolarized gas images

Tustison

Mata

2021

Preprint

View full text Add to dashboard Cite

Magnetic resonance imaging using hyperpolarized gases has made possible the novel visualization of airspaces, such as the human lung, which has advanced research into the growth, development, and pathologies of the pulmonary system. In conjunction with the innovations associated with image acquisition, multiple image analysis strategies have been proposed and refined for the quantification of such lung imaging with much research effort devoted to semantic segmentation, or voxelwise classification, into clinically-oriented categories based on ventilation levels. Given the functional nature of these images and the consequent sophistication of the segmentation task, many of these algorithmic approaches reduce the complex spatial image information to intensity-only considerations, which can be contextualized in terms of the intensity histogram. Although facilitating computational processing, this simplifying transformation results in the loss of important spatial cues for identifying salient image features, such as ventilation defects (a well-studied correlate of lung pathophysiology), as spatial objects. In this work, we discuss the interrelatedness of the most common approaches for histogram-based optimization of hyperpolarized gas lung imaging segmentation and demonstrate how certain assumptions lead to suboptimal performance, particularly in terms of measurement precision. In contrast, we illustrate how a convolutional neural network is optimized (i.e., trained) directly within the image domain to leverage spatial information. This image-based optimization mitigates the problematic issues associated with histogram-based approaches and suggests a preferred future research direction. Importantly, we provide the entire processing and evaluation framework, including the newly reported deep learning functionality, as open-source through the well-known Advanced Normalization Tools ecosystem (ANTsX).

show abstract

Section: Discussionmentioning

confidence: 99%

Image- vs. histogram-based considerations in semantic segmentation of pulmonary hyperpolarized gas images

Tustison

Mata

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Most studies demonstrate good results with two or three-fold down sampling. A full review of the use of machine learning super-resolution in medical imaging can be found [28].…”

Section: Image Restoration Methodsmentioning

confidence: 99%

Machine learning in Magnetic Resonance Imaging: Image reconstruction

et al. 2021

View full text Add to dashboard Cite

Magnetic Resonance Imaging (MRI) plays a vital role in diagnosis, management and monitoring of many diseases. However, it is an inherently slow imaging technique. Over the last 20 years, parallel imaging, temporal encoding and compressed sensing have enabled substantial speed-ups in the acquisition of MRI data, by accurately recovering missing lines of k-space data. However, clinical uptake of vastly accelerated acquisitions has been limited, in particular in compressed sensing, due to the time-consuming nature of the reconstructions and unnatural looking images. Following the success of machine learning in a wide range of imaging tasks, there has been a recent explosion in the use of machine learning in the field of MRI image reconstruction.A wide range of approaches have been proposed, which can be applied in k-space and/or image-space.Promising results have been demonstrated from a range of methods, enabling natural looking images and rapid computation.In this review article we summarize the current machine learning approaches used in MRI reconstruction, discuss their drawbacks, clinical applications, and current trends.

show abstract

“…pCLE HR images in the context of SR There has been extensive development in SR for natural images over last years [8], and the state-of-the-art methods are in the majority based on DL [9]. Many studies have examined DL-based SR in medical imaging as well [10], with considerable interest in Generative Adversarial Networks (GAN) for image reconstruction and denoising [11]. This section outlines the existing DL-based solutions presented in the literature for tackling specifically SR for pCLE with a focus on contending with the limitations in terms of availability of ground-truth HR data.…”

Section: Related Workmentioning

confidence: 99%

“…Previous studies on DL-based SR have primarily concentrated on using a default bicubic kernel with anti-aliasing as counterparts of real downscaling kernels. The NTIRE challenge [10], [16] aims at developing SR methods able to not only perform well on simulated images with a known bicubic kernel but also on "real" cases with unknown downscaling kernels. The results of the challenge demonstrated that the misestimation of the downscaling kernel affects the quality of super-resolved images.…”

Section: Downscaling Kernels For Srmentioning

confidence: 99%

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

Szczotka

Shakir

Clarkson

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth highresolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for groundtruth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope's irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts.

show abstract

A Review of the Deep Learning Methods for Medical Images Super Resolution Problems

Cited by 191 publications

References 26 publications

Image- vs. histogram-based considerations in semantic segmentation of pulmonary hyperpolarized gas images

Image- vs. histogram-based considerations in semantic segmentation of pulmonary hyperpolarized gas images

Machine learning in Magnetic Resonance Imaging: Image reconstruction

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

Contact Info

Product

Resources

About