Textural-Perceptual Joint Learning for No-Reference Super-Resolution Image Quality Assessment

Abstract

Image super-resolution (SR) has been widely investigated in recent years. However, it is challenging to fairly estimate the performances of various SR methods, as the lack of reliable and accurate criteria for perceptual quality. Existing SR image quality assessment (IQA) metrics usually concentrate on the specific kind of degradation without distinguishing the visual sensitive areas, which have no adaptive ability to describe the diverse SR degeneration situations. In this paper, we focus on the textural and structural degradation of image SR which acts as a critical role for visual perception, and design a dual stream network to jointly explore the textural and structural information for quality prediction, dubbed TSNet. By mimicking the human vision system (HVS) that pays more attention to the significant areas of the image, we develop the spatial attention mechanism to make the visual-sensitive areas more distinguishable, which improves the prediction accuracy. Feature normalization (F-Norm) is also developed to investigate the inherent spatial correlation of SR features and boost the network representation capacity. Experimental results show the proposed TSNet predicts the visual quality more accurate than the state-ofthe-art IQA methods, and demonstrates better consistency with the human's perspective. The source code will be made available at http://github.com/yuqing-liu-dut/NRIQA SR.

Residual scale attention network for arbitrary scale image super-resolution

“…In order to achieve better modeling ability, various neural architecture design strategies are proposed. Lu et al [13] deepen the depth of IQA models, while Zhao et al [14] and Liu et al [15] increase the width of IQA models, i.e., adopting dual-branch architectures. Besides, Zhang et al [16] introduce channel-spatial attention mechanisms into IQA models.…”

“…With the success of the first deep learning-based blind SR IQA measure, researchers shift their focus from feature engineering to neural architecture engineering. In the past few years, various neural architecture strategies including deeper network architecture [13], dual-branch framework [14], [15], and visual attention mechanisms [16], have been proposed to further improve the performance.…”