Deep Learning-Based Lighting Estimation for Indoor and Outdoor

Abstract

We propose a deep learning-based method that can estimate an appropriate lighting of both indoor and outdoor images. The method consists of two networks: Crop-to-PanoLDR network and LDR-to-HDR network. The Crop-to-PanoLDR network predicts a low dynamic range (LDR) environment map from a single partially observed normal field of view image, and the LDR-to-HDR network transforms the predicted LDR image into a high dynamic range (HDR) environment map which includes the high intensity light information. The HDR environment map generated through this process is applied when rendering virtual objects in the given image. The direction of the estimated light along with ambient light illuminating the virtual object is examined to verify the effectiveness of the proposed method. For this, the results from our method are compared with those from the methods that consider either indoor images or outdoor images only. In addition, the effect of the loss function, which plays the role of classifying images into indoor or outdoor was tested and verified. Finally, a user test was conducted to compare the quality of the environment map created in this study with those created by existing research.

Towards Generating Authentic Human-Removed Pictures in Crowded Places Using a Few-Second Video

“…Existing smartphone applications, such as Retouch-photos [1], Spectre [2], and Samsung object eraser [3], while potentially viable, require the users' manual processing, which could burden them. In addition, our preliminary study [4] uncovered that the quality of certain parts of the resulting image was degraded significantly. This may be attributed to the complexity of certain regions in which people should be removed or the movement of people to obtain the desired images.…”

Contrast Enhancement-Based Preprocessing Process to Improve Deep Learning Object Task Performance and Results

“…In addition, deep learning technology that uses a lot of computing resources has emerged due to the development of big data and hardware devices such as CPUs, GPUs, and TPUs. However, image data improvement technology that uses deep learning algorithms has the following disadvantages: (1) overfitting problems due to lack of training data; (2) generalization performance degradation problems due to biases in training data; and (3) speed reduction problems due to high amounts of computation and memory usage [13,14]. In particular, problems such as slowdown occur in unmanned vehicles, which require a small amount of computing resources [15].…”