Deep learning for pixel-level image fusion: Recent advances and future prospects

Liu, Yu; Chen, Xun; Wang, Zengfu; Wang, Z. Jane; Ward, Rabab K.; Wang, Xuesong

doi:10.1016/j.inffus.2017.10.007

Cited by 591 publications

(246 citation statements)

References 98 publications

Supporting

Mentioning

244

Contrasting

Unclassified

Order By: Relevance

“…However, deep learning in medical image registration has not been extensively studied until the past three to four years. Though several review papers on deep learning in medical image analysis have been published [73,93,96,105,106,121,132,182], there are very few review papers that are specific to deep learning in medical image registration [60]. The goal of this paper is to summarize the latest developments, challenges and trends in DL-based medical image registration methods.…”

Section: Introductionmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

View full text Add to dashboard Cite

This paper presents a review of deep learning (DL) based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potentials. We provided a comprehensive comparison among DL-based methods for lung and brain registration using benchmark datasets. Lastly, we analyzed the statistics of all the cited works from various aspects, revealing the popularity and future trend of DL-based medical image registration. 1 arXiv:1912.12318v1 [eess.IV] 27 Dec 2019 1. Summarize the latest developments in DL-based medical image registration. 2. Highlight contributions, identify challenges and outline future trends. 3. Provide detailed statistics on recent publications from different perspectives. 2 Deep Learning 2 Convolutional Neural NetworkConvolutional neural network (CNN) is a class of deep neural networks with regularized multilayer perceptron. CNN uses convolution operation in place of general matrix multiplication in simple neural networks. The convolutional filters and operations in CNN make it suitable for visual imagery signal processing. Because of its excellent feature extraction ability, CNN is one of the most successful models for image analysis. Since the breakthrough of AlexNet [79], many variants of CNN have been proposed and have achieved the-state-of-art performances in various image processing tasks. A typical CNN usually consists of multiple convolutional layers, max pooling layers, batch normalization layers, dropout layers, a sigmoid or softmax layer. In each convolutional layer, multiple channels of feature maps were extracted by sliding trainable convolutional kernels across the input feature maps. Hierarchical features with high-level abstraction are extracted using multiple convolutional layers. These feature maps usually go through multiple fully connected layer before reaching the final decision layer. Max pooling layers are often used to reduce the image sizes and to promote spatial invariance of the network. Batch normalization is used to reduce internal covariate shift among the training samples. Weight regularization and dropout layers are used to alleviate data overfitting. The loss function is defined as the difference between the predicted and the target output. CNN is usually trained by minimizing the loss via gradient back propagation using optimization methods. Many different types of network architectures have been proposed to improve the performance of CNN [93]. U-Net proposed by Ronneberger et al. is among one of the most used network architectures [120]. U-Net was originally used to perform neuronal structures segmentation. U-Net adopts symmetrical contractiv...

show abstract

Section: Introductionmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Parameters: In the next section we describe different applications to imaging data of Algorithm 1 and test its sensitivity to the weights (η, µ, γ), with Huberised smoothing of the L 1 gradient norm with fixed smoothing parameter ε = 0.05. For the iPiano algorithm we use the following parameters: the exit condition for the iPiano (with backtracking) scheme is chosen so as to satisfy either the relative error on the energy (1e-6) or the maximum number of iterations (10000), with at least few iterations performed so as to assure the inertial contribution; the exit condition for the nested primal-dual problem (19) is set to satisfy either the relative error on the primal-dual gap 1e-4 or the maximum number of iterations (10000).…”

Section: ) Derivatives Of O(u V)mentioning

confidence: 99%

“…This effect, is desirable in this application for enhancing the edges of the letters. We remark that despite the availability of few metrics assessing the quality of image fusion based either on image correlation [4] or on deep learning [19], we remark that for Cultural Heritage applications the validation of the optimal result is performed by expert users, so as to take into account not only image features, but possible other characteristics such as author style, historic period, illumination, etc.…”

Section: Multi-modal Fusion In Cultural Heritagementioning

confidence: 99%

Variational Osmosis for Non-Linear Image Fusion

Parisotto

Calatroni

Bugeau

et al. 2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

We propose a new variational model for nonlinear image fusion. Our approach incorporates the osmosis model proposed in and as an energy term in a variational model. The osmosis energy is known to realize visually plausible image data fusion. As a consequence, our method is invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision and parameter tuning and can encode prior information on the structure of the images to be fused. We develop a primaldual algorithm for solving this new image fusion model and we apply the resulting minimisation scheme to multi-modal image fusion for face fusion, colour transfer and some cultural heritage conservation challenges. Visual comparison to state-of-the-art proves the quality and flexibility of our method.

show abstract

“…The Linear Filters attained the transfer function as it assures superposition principle and that will be only presented in transfer function form. All the frequency components are phase shifted by -π/2 radians with the help of linear filter [10]. For all the frequencies the magnitude characteristics of the filter are 1 as the actual signals have affirmative and negative frequencies.…”

Section: Hilbert Transformmentioning

confidence: 99%

A Survey on Image Fusion Techniques for Image Enhancement in Digital Image Processing

Kaur¹,

Kaur²

2018

IJCA

View full text Add to dashboard Cite

Image fusion procedure is helpful to attain a single output image that contains required data or information by merging two distorted input images. Image fusion is done by pulling out all the necessary information from two images or more than two images after which the extorted information is combined into a distinct fused image. This fused image has improved superiority as compare to the input images. Image fusion is prepared by implementing particular techniques. Those particular schemes for image fusion are introduced in this paper. This paper offers an overview to the work that has been done by in past.

show abstract

Deep learning for pixel-level image fusion: Recent advances and future prospects

Cited by 591 publications

References 98 publications

Deep learning in medical image registration: a review

Deep learning in medical image registration: a review

Variational Osmosis for Non-Linear Image Fusion

A Survey on Image Fusion Techniques for Image Enhancement in Digital Image Processing

Contact Info

Product

Resources

About