A Deep Learning Approach to Mobile Camera Image Signal Processing

Abstract

The quality of the images obtained from mobile cameras has been an important feature for modern smartphones. The camera Image Signal Processing (ISP) is a significant procedure when generating high-quality images. However, the existing algorithms in the ISP pipeline need to be tuned according to the physical resources of the image capture, limiting the final image quality. This work aims at replacing the camera ISP pipeline with a deep learning model that can better generalize the procedure. A Deep Neural Network based on the UNet architecture was employed to process RAW images into RGB. Pre-processing stages were applied, and some resources for training were added incrementally. The results demonstrated that the test images were obtained efficiently, indicating that the replacement of traditional algorithms by deep models is indeed a promising path.

Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report

“…For this, a Zurich RAW to RGB dataset containing RAW-RGB image pairs from a mobile camera sensor and a high-end DSLR camera was collected. The proposed learned ISP reached the quality level of commercial ISP system of the Huawei P20 camera phone, and these results were further improved in [37,7,60,43,33]. In this challenge, we use a more advanced FujiFlim UltraISP dataset [27,23] and additional efficiency-related constraints on the developed solutions.…”