DeepFlash: Turning a flash selfie into a studio portrait

Abstract

We present a method for turning a flash selfie taken with a smartphone into a photograph as if it was taken in a studio setting with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in an ad-hoc acquisition campaign. Each pair consists of one photograph of a subject's face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend defects introduced by a close-up camera flash, such as specular highlights, shadows, skin shine, and flattened images.

Ambient Lighting Generation for Flash Images with Guided Conditional Adversarial Networks

“…Enhancement of low-illuminated areas (red), and estimation of natural skin and air tone of people (green). We compare with SRIE (Fu et al, 2016), LIME (Guo et al, 2017), and DeepFlash (Capece et al, 2019). we used a reduced set of the entire FAID dataset for our experiments. Finally, our custom dataset has 969 pairs of images for training and 116 for testing and all images were resized to 320 × 240 or 240 × 320 depending on their orientation.…”

“…The encoder part of the network represents the first 13 convolutional blocks of the VGG-16 (Simonyan and Zisserman, 2015), and the weights of the encoder are initialized with a pre-trained model for face-recognition (Parkhi et al, 2015). (Capece et al, 2019) implemented a pre-processing step before feed the inputs and targets to their architecture, each pair of images are filtered. After this step the encoder-decoder architecture is trained to learn a lowfrequency relationship between the flash and the ambient image.…”

“…Our quantitative results do not significantly outperform the stateof-the-art image enhancement methods, but at least shows improvements on the flash image enhancement task. (Fu et al, 2016), LIME (Guo et al, 2017), and DeepFlash (Capece et al, 2019).…”

“…Generation of ambient shadows (green), attenuation of overexposed areas (red), and sideways shadow removal (orange). We compare with SRIE (Fu et al, 2016), LIME (Guo et al, 2017), and DeepFlash (Capece et al, 2019). in Figure 4, where the illumination map created by LIME (Guo et al, 2017) conducts to brightening and overexposing the flash images. LIME (Guo et al, 2017), can not distinguish the natural color of dark objects and tend to illuminate them.…”

“…Our method attenuates the illumination that is close to the camera, and illuminates the underexposed regions at the same time. Since flash and ambient images represent the same scene, researchers (Capece et al, 2019) study the lighting normalization on a flash image by learning the relationship between both images to estimate a relationship between these pair of images, which is added to the respective flash image in a next step, thus, normalizing the illumination on flash images but maintaining high-frequency information. This approach is not effective to restore overexposed areas due to this region still needs to compute the final result.…”