A Hardware-Aware Network for Real-World Single Image Super-Resolutions

Abstract

Most single image super resolution (SISR) methods are developed on synthetic low resolution (LR) and high resolution (HR) image pairs, which are simulated by a predetermined degradation operation, such as bicubic downsampling. However, these methods only learn the inverse process of the predetermined operation, which fails to super resolve the real-world LR images, whose true formulation deviates from the predetermined operation. To address this, we propose a novel SR framework named hardware-aware super-resolution (HASR) network that first extracts hardware information, particularly the camera degradation information. The LR images are then super resolved by integrating the extracted information. To evaluate the performance of HASR network, we build a dataset named Real-Micron from real-world micron-scale patterns. The paired LR and HR images are captured by changing the objectives and registered using a developed registration algorithm. Transfer learning is implemented during the training of Real-Micron dataset due to the lack of amount of data. Experiments demonstrate that by integrating the degradation information, our proposed network achieves state-of-the-art performance for the blind SR task on both synthetic and real-world datasets.Impact Statement-The proposed HASR method has significant impact on various areas, such as enhancing the accurate inspection of manufactured products for quality control and enhancing the resolution of medical images to enable more accurate diagnosis and healthcare. Current SR solutions neglect the uniqueness of each imaging system, hence cannot produce accurate HR images across the different systems. Taking advantage of the known hardware information, HASR can differentiate lowresolution images across different imaging systems and produce HR images that are closer to the real-world scenario. Given sufficient training images, the proposed HASR method can overcome the physical optical limitation and generate higher quality images. The proposed method improves the overall performance by about 0.2 dB and 0.5 dB on the synthetic and the real-world datasets, respectively.