Programmable Triangulation Light Curtains

Wang, Jian; Bartels, Joseph; Whittaker, William; Sankaranarayanan, Aswin C.; Narasimhan, Srinivasa G.

doi:10.1007/978-3-030-01219-9_2

Cited by 31 publications

(20 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, it would be useful to have control over the projector's raster scanning behavior, and estimate 4D light transport from this illumination. Such programmability has recently been shown to be useful for performing disparity gating and capturing complex geometric light transport paths [61], [62]. Parametric analysis of epipolar and non-epipolar light using delay and exposure and possibly other hardware mechanisms can lead to future insights into the physical nature of these visual effects.…”

Section: Discussionmentioning

confidence: 99%

Programmable Non-Epipolar Indirect Light Transport: Capture and Analysis

Kubo

Jayasuriya

Iwaguchi

et al. 2021

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

The decomposition of light transport into direct and global components, diffuse and specular interreflections, and subsurface scattering allows for new visualizations of light in everyday scenes. In particular, indirect light contains a myriad of information about the complex appearance of materials useful for computer vision and inverse rendering applications. In this paper, we present a new imaging technique that captures and analyzes components of indirect light via light transport using a synchronized projector-camera system. The rectified system illuminates the scene with epipolar planes corresponding to projector rows, and we vary two key parameters to capture plane-to-ray light transport between projector row and camera pixel: (1) the offset between projector row and camera row in the rolling shutter (implemented as synchronization delay), and (2) the exposure of the camera row. We describe how this synchronized rolling shutter performs illumination multiplexing, and develop a nonlinear optimization algorithm to demultiplex the resulting 3D light transport operator. Using our system, we are able to capture live short and long-range non-epipolar indirect light transport, disambiguate subsurface scattering, diffuse and specular interreflections, and distinguish materials according to their subsurface scattering properties. In particular, we show the utility of indirect imaging for capturing and analyzing the hidden structure of veins in human skin.

show abstract

Section: Discussionmentioning

confidence: 99%

Programmable Non-Epipolar Indirect Light Transport: Capture and Analysis

Kubo

Jayasuriya

Iwaguchi

et al. 2021

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The latter refers to approaches that select the best sensing action for specific tasks such as object instance classification [29,11,10,25] and 3D reconstruction [15,16,26,9]. Light curtains were introduced in prior work [27,5] as an adaptive depth sensor. Prior work has also explored the use of light curtains.…”

Section: A Active Perception and Light Curtainsmentioning

confidence: 99%

“…Programmable light curtains [27,5,1] are a recently developed sensor for controllable depth sensing. "Light curtains" can be thought of as virtual surfaces placed in the environment that detect points on objects intersecting this surface.…”

Section: Background On Light Curtainsmentioning

confidence: 99%

“…An alternative approach is to use active perception [3,4], where only the important and required parts of the scene are accurately sensed, by actively guiding a controllable sensor in an intelligent manner. Specifically, a programmable light curtain [27,5,1] is a light-weight controllable sensor that detects objects intersecting any user-specified 2D vertically ruled surface (or a 'curtain'). Because they use an ordinary rolling shutter camera, light curtains combine the best of both worlds of passive cameras (high spatial-temporal resolution and lower cost) and LiDARs (accurate detection along the 2D curtain and robustness to scattered media like smoke/fog).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Active Safety Envelopes using Light Curtains with Probabilistic Guarantees

Ancha¹,

Pathak²,

Narasimhan³

et al. 2021

Robotics: Science and Systems XVII

View full text Add to dashboard Cite

To safely navigate unknown environments, robots must accurately perceive dynamic obstacles. Instead of directly measuring the scene depth with a LiDAR sensor, we explore the use of a much cheaper and higher resolution sensor: programmable light curtains. Light curtains are controllable depth sensors that sense only along a surface that a user selects. We use light curtains to estimate the safety envelope of a scene: a hypothetical surface that separates the robot from all obstacles. We show that generating light curtains that sense random locations (from a particular distribution) can quickly discover the safety envelope for scenes with unknown objects. Importantly, we produce theoretical safety guarantees on the probability of detecting an obstacle using random curtains. We combine random curtains with a machine learning based model that forecasts and tracks the motion of the safety envelope efficiently. Our method accurately estimates safety envelopes while providing probabilistic safety guarantees that can be used to certify the efficacy of a robot perception system to detect and avoid dynamic obstacles. We evaluate our approach in a simulated urban driving environment and a real-world environment with moving pedestrians using a light curtain device and show that we can estimate safety envelopes efficiently and effectively. 1

show abstract

“…Third, probing techniques isolate light following paths whose endpoints satisfy specific correspondences between a two-dimensional source and a two-dimensional sensor [O'Toole et al 2012]. These can be epipolar [O'Toole et al 2014b], disparity [O'Toole et al 2012;Wang et al 2018], or plane-ray [Kubo et al 2018;Liu et al 2020] correspondences, as well as their logical complements [O'Toole et al 2015]. Finally, there exist techniques that implement hybrids between different types of decompositions, for example by combining probing and transient imaging [Achar et al 2017;Gkioulekas et al 2015;Wang et al 2018].…”

Section: Related Workmentioning

confidence: 99%

Interferometric transmission probing with coded mutual intensity

2020

View full text Add to dashboard Cite

We introduce a new interferometric imaging methodology that we term interferometry with coded mutual intensity, which allows selectively imaging photon paths based on attributes such as their length and endpoints. At the core of our methodology is a new technical result that shows that manipulating the spatial coherence properties of the light source used in an interferometric system is equivalent, through a Fourier transform, to implementing light path probing patterns. These patterns can be applied to either the coherent transmission matrix, or the incoherent light transport matrix describing the propagation of light in a scene. We test our theory by building a prototype inspired by the Michelson interferometer, extended to allow for programmable phase and amplitude modulation of the illumination injected in the interferometer. We use our prototype to perform experiments such as visualizing complex fields, capturing direct and global transport components, acquiring light transport matrices, and performing anisotropic descattering, both in steady-state imaging and, by combining our technique with optical coherence tomography, in transient imaging. CCS Concepts: • Computing methodologies → Computational photography.

show abstract

Programmable Triangulation Light Curtains

Cited by 31 publications

References 19 publications

Programmable Non-Epipolar Indirect Light Transport: Capture and Analysis

Programmable Non-Epipolar Indirect Light Transport: Capture and Analysis

Active Safety Envelopes using Light Curtains with Probabilistic Guarantees

Interferometric transmission probing with coded mutual intensity

Contact Info

Product

Resources

About