Evaluation of Image Fusion Performance with Visible Differences

Abstract

Multisensor signal-level image fusion has attracted considerable research attention recently. Whereas it is relatively straightforward to obtain a fused image, e.g. a simple but crude method is to average the input signals, assessing the performance of fusion algorithms is much harder in practice. This is particularly true in widespread "fusion for display" applications where multisensor images are fused and the resulting image is presented to a human operator. As recent studies have shown, the most direct and reliable image fusion evaluation method, subjective tests with a representative sample of potential users are expensive in terms of both time/effort and equipment required. This paper presents an investigation into the application of the Visible signal Differences Prediction modelling, to the objective evaluation of the performance of fusion algorithms. Thus given a pair of input images and a resulting fused image, the Visual Difference Prediction process evaluates the probability that a signal difference between each of the inputs and the fused image can be detected by the human visual system. The resulting probability maps are used to form objective fusion performance metrics and are also integrated with more complex fusion performance measures. Experimental results indicate that the inclusion of visible differences information in fusion assessment yields metrics whose accuracy, with reference to subjective results, is superior to that obtained from the state of the art objective fusion performance measures.

Subjective tests for image fusion evaluation and objective metric validation

“…Qu et al [17] meanwhile proposed a metric based on mutual information, MI that considers only global image statistics in its evaluation. Finally, in [19] a number of fusion evaluation metrics based on the principle of visual differences was proposed the most successful of which (VDA) evaluates fusion performance as the total area affected by visible differences in the fused image as compared to the inputs. In order for such metrics to be truly applicable however, they need to be validated against a known ground truth, a set of evaluated fused images, such as those contained in the presented subjective tests.…”

“…Objective fusion evaluation metrics that require no display equipment or complex organisation of an audience are a far easier computed based alternative and a handful have already been proposed in the literature [5][6][7][16][17][18][19][20]. Of particular interest are the fully automatic, blind evaluation objective fusion metrics [17][18][19][20] that evaluate fusion without the ground truth in the form of an ideally fused image as is needed for RMSE error evaluation of multi-focus fusion [5,6]. Such metrics consider only the input images and the fused and produce a single numerical score that indicates the success of the fusion process.…”

“…A number of evaluation trials focusing on various aspects of fusion performance and different scenarios (applications) were reported in the literature [2,3,[11][12][13][14][15]. Despite the difficulties in obtaining good validation data however, a small number of objective fusion performance metrics have also emerged [17][18][19][20]. These metrics are by enlarge theoretically based, however no effort has so far been made to evaluate them independently.…”

“…by simply averaging them, assessing the performance of fusion algorithms is much harder in practice. Consequently, whereas a plethora of image fusion algorithms have appeared in the literature only a handful of publications deals with its evaluation [11][12][13][14][15][16][17][18][19]. One of the main issues that has limited research in this area from its onset has been a distinct lack of image data both to test advanced algorithms and validate their performance relative to 1566 one another and to formulate, test and validate efficient objective performance metrics.…”

A human perception inspired quality metric for image fusion based on regional information

“…However, there is no established direct relationship between these evaluation measures and the real perceptual results of humans. In [8], a Visual Difference Predictor (VDP) [9] based metric is proposed, where the fusion performance is measured in terms of the average probability of noticing a difference between the inputs and the fused image.…”