Real-Time Under-Display Cameras Image Restoration and HDR on Mobile Devices

Abstract

Multiple low-vision tasks such as denoising, deblurring and super-resolution depart from RGB images and further reduce the degradations, improving the quality. However, modeling the degradations in the sRGB domain is complicated because of the Image Signal Processor (ISP) transformations. Despite of this known issue, very few methods in the literature work directly with sensor RAW images. In this work we tackle image restoration directly in the RAW domain. We design a new realistic degradation pipeline for training deep blind RAW restoration models. Our pipeline considers realistic sensor noise, motion blur, camera shake, and other common degradations. The models trained with our pipeline and data from multiple sensors, can successfully reduce noise and blur, and recover details in RAW images captured from different cameras. To the best of our knowledge, this is the most exhaustive analysis on RAW image restoration. Code available at https://github.com/mv-lab/AISP

Recent Advances in MicroLED Displays with Integrated Image Capture

“…The computation challenge appears to be similar to that for micro-camera array imaging, because low latency, video-rate image restoration is required. 9 Further, the high-transparency requirement drives minimizing the microLED die's size, thereby reducing the power efficiency of the display.…”

“…However, disadvantages include substantially increasing the display thickness if a single‐lens camera is used, and significant real‐time image enhancement computation is required to correct for the display “screen door” of the image capture system. The computation challenge appears to be similar to that for micro‐camera array imaging, because low latency, video‐rate image restoration is required 9 . Further, the high‐transparency requirement drives minimizing the microLED die's size, thereby reducing the power efficiency of the display.…”