Unification of image fusion and super-resolution using jointly trained dictionaries and local information contents

Abstract

Multi-modal imaging requires image fusion to combine advantages of different types of sensors and requires superresolution (SR) because of limited spatial resolution of source images. In this study, a novel framework is proposed for unification of image SR and the fusion process to obtain a high-resolution (HR)-fused image from a set of low-resolution (LR) multi-modal images. The jointly trained dictionaries of LR patches and corresponding HR patches are used for sparse representation of LR source image patches and HR-fused image patches, respectively. The sparse coefficients vectors for corresponding patches of source LR images are determined by using orthogonal matching pursuit and a local information content-based metric is employed to fuse these sparse coefficients. The corresponding HR-fused image patch is obtained by combining elements of the HR dictionary as per the fused coefficients of the LR image patches. The experimental results on sets of multi-modal images exhibited that the proposed method outperformed the existing fusion and SR techniques in terms of visual quality and image fusion quality metrics.

Pan‐sharpening based on weighted red black wavelets

“…If the injection of the details into an MS image without testing the similarity between the coefficients of the PAN and the MS images, it causes the insertion of noise into the MS images [18]. In order to obtain pan‐sharpened images with fidelity colour and noiseless enhanced details, it is necessary to inject the details according to the local information of the data [19]. By using local approach, methods such as [5, 20, 12] have been proved to produce stable results compared with other methods [8].…”

Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review

“…The main idea of the method is to assume that the upsampled low-resolution (LR) and high-resolution (HR) image patch pairs share the same sparse coefficients with respect to their own dictionaries. Recently, this idea was applied to multi-sensor image fusion [25], [57], [58] as well as pan-sharpening [26], [59].…”

Simultaneous image fusion and denoising with adaptive sparse representation

“…The fused image is reconstructed with all the fused sparse vectors and the dictionary. Many improved SR-based fusion methods [21][22][23][24][25] have been proposed since this fundamental work appeared. These publications show that the SR-based methods, which own clear advantages over traditional multi-scale transform (e.g.…”

“…Most recently, some new TDF methods based on multi-scale filtering have been introduced, such as the multi-scale edge-preserving decomposition-based method [15] and the neighbour distance (ND)-based method [16]. Sparse representation (SR) has been successfully employed in many image processing applications including denoising [17], super-resolution [18,19] and fusion [20][21][22][23][24][25]. The SR-based fusion methods can also be classified into the TDF methods since the activity level of source images is measured in the sparse domain.…”