Adaptive Image Sampling Using Deep Learning and Its Application on X-Ray Fluorescence Image Reconstruction

Abstract

This paper presents an adaptive image sampling algorithm based on Deep Learning (DL). It consists of an adaptive sampling mask generation network which is jointly trained with an image inpainting network. The sampling rate is controlled by the mask generation network, and a binarization strategy is investigated to make the sampling mask binary. In addition to the image sampling and reconstruction process, we show how it can be extended and used to speed up raster scanning such as the X-Ray fluorescence (XRF) image scanning process. Recently XRF laboratory-based systems have evolved into lightweight and portable instruments thanks to technological advancements in both X-Ray generation and detection. However, the scanning time of an XRF image is usually long due to the long exposure requirements (e.g., 100µs − 1ms per point). We propose an XRF image inpainting approach to address the long scanning times, thus speeding up the scanning process, while being able to reconstruct a high quality XRF image. The proposed adaptive image sampling algorithm is applied to the RGB image of the scanning target to generate the sampling mask. The XRF scanner is then driven according to the sampling mask to scan a subset of the total image pixels. Finally, we inpaint the scanned XRF image by fusing the RGB image to reconstruct the full scan XRF image. The experiments show that the proposed adaptive sampling algorithm is able to effectively sample the image and achieve a better reconstruction accuracy than that of existing methods.

Adaptive Illumination based Depth Sensing using Deep Superpixel and Soft Sampling Approximation

“…Finally, these sampling techniques are all applied to the same modality (RGB or grey scale image). Recently, Dai et al [8] applied DL technique to the adaptive sampling problem. The adaptive sampling network is jointly optimized with the image inpainting network.…”

“…All of the above DL based sampling methods predict a per pixel sampling probability [8,21,22] or a sampling number [31]. Good sampling performance has not been demonstrated under extreme low sampling rates (< 1%).…”

Efficient Data Optimisation for Harmonic Inpainting with Finite Elements

“…Although these approaches are relatively simple, they can achieve very good quality if the inpainting data are carefully optimised [4,9,13,21,25]. Their quality also exceeds the one reported for recent neural network approaches for sparse inpainting [7].…”

Image = Structure + Few Colors

“…For example, in [28], an efficient image down-sampling and up-sampling technique based on interpolation is developed. Other approaches include [8] and [13]. Besides down-sampling, upsampling is equally important for applications like image super-resolution, image enhancement and denoising.…”