End-to-End Full Projector Compensation

Huang, Bingyao; Sun, Tao; Ling, Haibin

doi:10.1109/tpami.2021.3050124

Cited by 16 publications

(27 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once trained, the network generates a projection image from an original image such that the projected result does not suffer from photometric distortions caused by the spatially varying surface reflectance properties. The network was improved to compensate for both photometric and geometric distortions [18,21] as well as for global illumination effects [20]. These networks outperform classical technologies with regard to radiometric compensation.…”

Section: Deep Learning For Radiometric Compensationmentioning

confidence: 99%

Online Projector Deblurring Using a Convolutional Neural Network

Kageyama

Iwai

Sato

2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Section: Deep Learning For Radiometric Compensationmentioning

confidence: 99%

Online Projector Deblurring Using a Convolutional Neural Network

Kageyama

Iwai

Sato

2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

“…Huang and Ling [12] proposed a unified framework (CompenNet++) that handles both geometric and color correction by joining a newly designed submodule (WarpingNet) for geometric calibration into CompenNet [13] for photometric correction. Later, Huang et al [16] optimized the CompenNet++ and presented a refined deep neural network (CompenNeSt++) with a reduced number of learnable parameters. Also, Huang and Ling [14] presented a sophisticated neural network that models light interactions with a projection surface and addressed image-based relighting, photometric correction, and estimation for depth and normal estimation simultaneously.…”

Section: Geometric and Photometric Correctionmentioning

confidence: 99%

“…For such approximation, existing approaches exploited simple hand-crafted functions such as matrix multiplication [3], [9], [23], [33], [36] and TPS [10], [11]. Also, the recent learning-based techniques [12], [16] approximate the unknown with a deep neural network. Nevertheless, their approximation quality can be degraded, as their approximation functions did not consider a complete light transport process.…”

Section: A Technical Challenges and Our Motivationmentioning

confidence: 99%

“…A recent alternative to the classical approaches is to employ a deep neural network that infers the optimal projector input image to match an output projection image with a target image [12], [16]. While the state-of-the-art learning-based methods demonstrated outstanding results compared to classical models by taking both geometric and photometric correction into account, their visual artifacts (i.e., a discrepancy VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Projector Compensation Framework Using Differentiable Rendering

2022

View full text Add to dashboard Cite

Projection mapping is a widely adopted technique in various applications. A typically projector-camera system consists of a projector and camera pair where a projector emits an application-specific input image onto a surface, and the camera captures the projected output image. Given an ideal projection surface with white-colored planar geometries, the camera (or a viewer) can observe the projected output image without any visual distortion. The surface, however, can have arbitrary shapes and textured colors in practice, and it often introduces visually distracting artifacts to the output image. It results in lowering the viewing experience drastically. We propose a projector compensation framework that adjusts the projector input image so that the camera can see the projected output image with a much-reduced distortion.Our key contribution is to model the real projection mapping process with a virtual but controllable light simulation and optimize the projector input using differentiable rendering. We demonstrate that our new framework produces a more accurate output than state-of-the-art methods given complex projection surfaces.

show abstract

“…Unfortunately, most of the existing full compensation methods [1] [2] are not suitable for continuous display since they need to waste extra frames projecting extraneous content, which significantly interrupts the display process and degrades the viewing experience of observers. Others [3] [4] [5] [6] can obtain the geometric mapping and remove the effects of underlying textured surfaces from the projected natural images, but requires unacceptable computational time(usually more than ten minutes) for full compensation and still needs a white sampling image to identify reflectance. Therefore, challenges still remain around the issues of practical full compensation.…”

Section: Introductionmentioning

confidence: 99%

Efficient Full Projector Compensation using Natural Images

Li¹

2022

Preprint

View full text Add to dashboard Cite

<p> Practical full projector compensation allows the projection display to quickly adapt to textured projection surfaces and unexpected movement without interrupting the display procedure or requiring unacceptable waiting time. To achieve this by a projector and a RGB camera without adding any extra devices, a possible solution is correcting both color and geometry by directly capturing and analyzing the projected natural images content. In this work, we cast the full projector compensation as a non-negative constrained optimization problem, and present the first optimization-based framework that can handle both geometric calibration and radiometric compensation for a Projector-camera system(Procams) using only a few sampling images. The framework not only takes advantage of the ability of the deep neural network to approximate the channel-dependent response function of the projector, but also provides a flexible solution to adapt to the change of geometry and reflectance caused by movement. Benefit from the optimization-based scheme, our method can guarantee both accurate color calculation and efficient movement and reflectance estimation in a continuous projection display procedure. The experimental results show our method outperforms other state-of-the-art end-to-end full projector compensation methods with better-displayed image quality, less computational time, smaller memory consumption, more geometric accuracy, and far more compact network architecture. </p>

show abstract

End-to-End Full Projector Compensation

Cited by 16 publications

References 60 publications

Online Projector Deblurring Using a Convolutional Neural Network

Online Projector Deblurring Using a Convolutional Neural Network

Projector Compensation Framework Using Differentiable Rendering

Efficient Full Projector Compensation using Natural Images

Contact Info

Product

Resources

About