Dual-Stage Super-Resolution for Edge Devices

Abstract

To reduce memory usage, edge devices such as TVs use super-resolution(SR) with dedicated hardware networks. Dedicated hardware has the disadvantage of being difficult to change and difficult to improve performance. We propose a dual-structured SR system that can extend this dedicated hardware, suggesting a way to improve the restoration performance for compression with minimal change in hardware. Edge devices such as general TVs deal with moving videos rather than still images. In this case, the video includes video compression, and a preprocessing step of restoring deterioration is useful. We propose as the main idea that these steps can be handled while removing only pixel shuffle layers from existing networks. Our proposed SR method consists of two stages: 1) restoring from the video compression without changing the size of LR images and 2) increasing the resolution. The first stage can be learned by reducing the difference between video-LR (Low-Resolution video images with codec degradation) and downscaled-HR (video images made without codec degradation by simply reducing the size of High-Resolution video). The second stage, resolution enhancement, performs the same task as the traditional SISR task, except that it focuses on restoring the output of the first stage rather than a downscaled-HR. Our new dataset for this processing, HD2UHD, consists of (video-LR, downscaled-HR, and HR) tuples. We also propose a new scheme of input distillation that utilizes video-LR and downscaled-HR at the same time.