ProDebNet: projector deblurring using a convolutional neural network

Abstract

Projection blur can occur in practical use cases that have non-planar and/or multi-projection display surfaces with various scattering characteristics because the surface often causes defocus and subsurface scattering. To address this issue, we propose ProDebNet, an end-to-end real-time projection deblurring network that synthesizes a projection image to minimize projection blur. The proposed method generates a projection image without explicitly estimating any geometry or scattering characteristics of the projection screen, which makes real-time processing possible. In addition, ProDebNet does not require real captured images for training data; we design a “pseudo-projected” synthetic dataset that is well-generalized to real-world blur data. Experimental results demonstrate that the proposed ProDebNet compensates for two dominant types of projection blur, i.e., defocus blur and subsurface blur, significantly faster than the baseline method, even in a real-projection scene.

Online Projector Deblurring Using a Convolutional Neural Network

“…Other researchers achieved projector deblurring with fewer artifacts by applying a constrained optimization technique [52] or an inverse light transport matrix technique [50], though they were computationally expensive. Kageyama et al balanced the trade-off between the deblurring accuracy and the computational complexity using a deep neural network (DNN) [28]. Grosse et al also balanced the trade-off by applying a coded aperture to the projector optics, which preserves the high-frequency components of a projected image more than do normal circular apertures and, consequently, reduces the ringing artifacts caused by Wiener filtering [13].…”

“…In these aforementioned techniques, the pixel-dependent PSFs must be estimated for the deconvolution. The estimation is done by projecting either dot patterns [9,13,50,52] or original images (i.e., target images) [28,38] in advance. Therefore, unblurred images cannot be continuously displayed on a moving projection surface, where the PSFs vary in time.…”

“…Previous studies estimated the PSFs by projecting a calibration pattern, which is either a spatial code pattern such as a dot pattern [9,13,52] or an original target image [28,38], and capturing the projected results using a camera. The estimated PSFs are valid for as long as the projector and the surface do not move.…”

CompenNet++: End-to-End Full Projector Compensation

“…In theory, the projector compensation process is a very complicated nonlinear function involving the camera and the projector sensor radiometric responses [38], lens distortion/vignetting [30], perspective transformations [23], [63], surface material reflectance [21], [41], defocus [31], [59], [61] and inter-reflection [53]. A great amount of effort has been dedicated to designing practical and accurate compensation models, which can be roughly categorized into two types: full compensation [4], [17], [44], [49], [51], [52], [58] and partial ones [1], [3], [13], [15], [20], [35], [38], [48], [53].…”

Super-Resolution and Large Depth of Field Model for Optical Microscope Imaging

“…With the rapid development of deep learning in computer vision and image processing, it is possible to use deep learning algorithms to obtain a clear image of the specimen with a large DOF. For example, the super-resolution (SR) and deblurring algorithms are used to improve the resolution of images [1][2][3][4][5][6][7][8][9]. Recently, the deep learning single-image super-resolution (SISR) model [10][11][12] and convolutional neural network (CNN) model [13] have been applied to improve the resolution and definition in microscope images, which indicates that they have great potential in microscopes.…”