Projector Compensation Framework Using Differentiable Rendering

Abstract

Projection mapping is a widely adopted technique in various applications. A typically projector-camera system consists of a projector and camera pair where a projector emits an application-specific input image onto a surface, and the camera captures the projected output image. Given an ideal projection surface with white-colored planar geometries, the camera (or a viewer) can observe the projected output image without any visual distortion. The surface, however, can have arbitrary shapes and textured colors in practice, and it often introduces visually distracting artifacts to the output image. It results in lowering the viewing experience drastically. We propose a projector compensation framework that adjusts the projector input image so that the camera can see the projected output image with a much-reduced distortion.Our key contribution is to model the real projection mapping process with a virtual but controllable light simulation and optimize the projector input using differentiable rendering. We demonstrate that our new framework produces a more accurate output than state-of-the-art methods given complex projection surfaces.

Projection mapping technologies: A review of current trends and future directions

“… 81 ) Other researchers used a differentiable rendering framework in radiometric compensation. 82 ) Handling high-resolution images typically requires long training times and involves high memory costs. Wang et al mitigated this issue by incorporating a sampling scheme into the network and introducing attention blocks.…”

iCatcher: A Neural Network Approach for Automated Coding of Young Children’s Eye Movements

“…They are however constrained to a single camera and single projector location, and project to (roughly) planar surfaces, since the warping field is limited in its application for difficult geometries. Park et al [27] posed the calibration and compensation as an inverse rendering problem in a similar manner to our solution, but again are constrained to a single projector-camera pair location and require a RGBD camera. In many of these techniques, the decomposition of the scene into interpretable physical quantities is key to high performance.…”