Camera-based detection and removal of shadows from interactive multiprojector displays

Abstract

Front-projection displays are a cost-effective and increasingly popular method for large format visualization and immersive rendering of virtual models. New approaches to projector tiling, automatic calibration, and color balancing have made multiprojector display systems feasible without undue infrastructure changes and maintenance. As a result, front-projection displays are being used to generate seamless, visually immersive worlds for virtual reality and visualization applications with reasonable cost and maintenance overhead. However, these systems suffer from a fundamental problem: Users and other objects in the environment can easily and inadvertently block projectors, creating shadows on the displayed image. Shadows occlude potentially important information and detract from the sense of presence an immersive display may have conveyed. We introduce a technique that detects and corrects shadows in a multiprojector display while it is in use. Cameras observe the display and compare observations with an expected image to detect shadowed regions. These regions are transformed to the appropriate projector frames, where corresponding pixel values are increased and/or attenuated. In display regions where more than one projector contributes to the image, shadow regions are eliminated.

Defocus Blur Correcting Projector-Camera System

“…This color mapping can be described as a CTF (Color Transfer Function) [19]. A pixel value in the projected image I p (x, y) is changed to a pixel value in the captured image I c (x, y) by using calculated CTF C,…”

Efficient object shape recovery via slicing planes

“…The ground plane contains squares which provides us with four measures required to estimate the homography from the images to the reference image and to the ground plane in the object space. Two pens oriented in the normal direction of the plane provides both the vanishing point in the direction of Z axis and the scale factor s given in equation (2). The dataset of the toyexperiment contains eleven images of the toy in which the objects parts are partially or fully occluded.…”

“…The implied 3D scene information through the reference plane and its homography transform in the images have been used in various applications, including but not limited to the tracking of people [9], shadow removal [2], and detection of low-lying objects [8]. Most of these techniques have been conceptually proven to have robust performance in practical scenarios.…”

A Theory for Photometric Self-Calibration of Multiple Overlapping Projectors and Cameras

“…Geometric calibration has been discussed in various papers [4,13,15,8,16,14] and will not be covered here. Most of the related work on photometric calibration falls into a number of categories:…”

“…Camera-observed intensities are recorded for a range of projector input intensities and used as a look-up table (LUT) [18], or modeled with parametric curves [8]. Intermediate projectorto-screen and screen-to-camera intensity transfer functions are however not recovered.…”