Demultiplexing illumination via low cost sensing and nanosecond coding

Abstract

Several computer vision algorithms require a sequence of photographs taken in different illumination conditions, which has spurred development in the area of illumination multiplexing. Various techniques for optimizing the multiplexing process already exist, but are geared toward regular or high speed cameras. Such cameras are fast, but code on the order of milliseconds. In this paper we propose a fusion of two popular contexts, time of flight range cameras and illumination multiplexing. Time of flight cameras are a low cost, consumer-oriented technology capable of acquiring range maps at 30 frames per second. Such cameras have a natural connection to conventional illumination multiplexing strategies as both paradigms rely on the capture of multiple shots and synchronized illumination. While previous work on illumination multiplexing has exploited coding at millisecond intervals, we repurpose sensors that are ordinarily used in time of flight imaging to demultiplex via nanosecond coding strategies.

Signal Processing for Time-of-Flight Imaging Sensors: An introduction to inverse problems in computational 3-D imaging

“…With the advent of 3D sensing technology (most notably, the Microsoft Kinect XBox One) we can now replace room sized apparatus [10], moving sensors, and raster scan systems [14] by miniaturized, cost-effective, real-time, and full-frame ToF systems. Computational ToF imaging has already found a plethora of applications in, for example, ultra-fast imaging [15], [16], non line-of-sight imaging [17], imaging through scattering media, [18] and colored ToF imaging [19]. Outside of the computational imaging and human-computer interaction communities, an important application area is health care technology [20] and bio-imaging [21].…”

Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

“…While yielding high resolution depth maps, single frequency TOF suffers from limitations including phase wrapping ambiguity and multipath interference caused by translucent objects and scattering media. Proposed techniques to overcome these limitations include phase unwrapping with multifrequency methods [29], global/direct illumination separation [19,41], deblurring and superresolution [42], and mitigating multipath interference with post-processing algorithms [3,4]. Recently, new temporal coding patterns for these sensors help resolve multiple optical paths to enable seeing light in flight and looking through turbid media [12,13,21].…”

A light transport model for mitigating multipath interference in Time-of-flight sensors

“…The recent release of the TOF-based Kinect sensor [47] will result in the wide-spread adaption of TOF cameras for various traditional and non-traditional applications [35,23,31,34,24,2,44] in computer vision, graphics and physics. In this paper we restrict our discussion to continuous-wave TOF cameras, which calculate depth by measuring the phase difference between an emitted and received optical signal.…”