Video Relighting Using Infrared Illumination

Wang, Oliver; Davis, James; Chuang, Erika; Rickard, Ian; Mesa, Krystle de; Dave, Chirag

doi:10.1111/j.1467-8659.2008.01124.x

Cited by 15 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several techniques have used ratio images to perform image relighting [28,31,40]. As opposed to previous uses, our RGB shading-ratio images take into account the artistic use of colors to encode shading.…”

Section: Image-based Methodsmentioning

confidence: 98%

Artistic relighting of paintings and drawings

Henz

Oliveira

2015

Vis Comput

View full text Add to dashboard Cite

We present a practical solution to the problem of subject relighting in paintings and drawings. Our interactive technique uses 3-D shading proxies and can be applied to objects with arbitrary geometries. Given a user-provided guess for the shading of an object in a painting/drawing and its corresponding target shading, we refine them using shading-color correlation and a multi-scale scheme. These refined shadings are then used to create a multi-channel shading-ratio image to perform relighting, while taking into account the colors used by the artists to convey shading information. We demonstrate the effectiveness of our solution on a variety of artistic styles, including paintings with strong brush strokes and unconventional shading encodings, drawings, and other types of artwork. Our method is the first to perform relighting of paintings and drawings and, in addition to relighting, can transfer smooth normal and depth maps from 3-D proxies to images.

show abstract

Section: Image-based Methodsmentioning

confidence: 98%

Artistic relighting of paintings and drawings

Henz

Oliveira

2015

Vis Comput

View full text Add to dashboard Cite

show abstract

“…A NIR (and/or ultraviolet) dark flash image can be used to denoise a color image captured in low visible light conditions [22], or serve as a guide for correcting motion blur [47]. Techniques have also been developed that employ controlled NIR lighting to simulate better visible lighting in real-time video communication systems [43,18]. We see these as compelling potential applications of this work.…”

Section: Monocular Depth Estimationmentioning

confidence: 98%

A Dark Flash Normal Camera

Xia,

Lawrence,

Achar

2020

Preprint

View full text Add to dashboard Cite

Casual photography is often performed in uncontrolled lighting that can result in low quality images and degrade the performance of downstream processing. We consider the problem of estimating surface normal and reflectance maps of scenes depicting people despite these conditions by supplementing the available visible illumination with a single near infrared (NIR) light source and camera, a socalled "dark flash image". Our method takes as input a single color image captured under arbitrary visible lighting and a single dark flash image captured under controlled front-lit NIR lighting at the same viewpoint, and computes a normal map, a diffuse albedo map, and a specular intensity map of the scene. Since ground truth normal and reflectance maps of faces are difficult to capture, we propose a novel training technique that combines information from two readily available and complementary sources: a stereo depth signal and photometric shading cues. We evaluate our method over a range of subjects and lighting conditions and describe two applications: optimizing stereo geometry and filling the shadows in an image.

show abstract

“…Video Relighting. Some methods [52,20,37] use IR and LEDs around the screen and web camera to provide acceptable lighting for video conferencing. Li et al [25] create free-viewpoint relighting video using multi-view reconstruction under general illumination, while Richardt et al [35] add video effects like relighting using RGBZ video cameras.…”

Section: Related Workmentioning

confidence: 99%

Neural Video Portrait Relighting in Real-time via Consistency Modeling

Zhang¹,

Zhang²,

Wu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Video portraits relighting is critical in user-facing human photography, especially for immersive VR/AR experience. Recent advances still fail to recover consistent relit result under dynamic illuminations from monocular RGB stream, suffering from the lack of video consistency supervision. In this paper, we propose a neural approach for realtime, high-quality and coherent video portrait relighting, which jointly models the semantic, temporal and lighting consistency using a new dynamic OLAT dataset. We propose a hybrid structure and lighting disentanglement in an encoder-decoder architecture, which combines a multi-task and adversarial training strategy for semantic-aware consistency modeling. We adopt a temporal modeling scheme via flow-based supervision to encode the conjugated temporal consistency in a cross manner. We also propose a lighting sampling strategy to model the illumination consistency and mutation for natural portrait light manipulation in realworld. Extensive experiments demonstrate the effectiveness of our approach for consistent video portrait light-editing and relighting, even using mobile computing.

show abstract

Video Relighting Using Infrared Illumination

Cited by 15 publications

References 30 publications

Artistic relighting of paintings and drawings

Artistic relighting of paintings and drawings

A Dark Flash Normal Camera

Neural Video Portrait Relighting in Real-time via Consistency Modeling

Contact Info

Product

Resources

About