Single Image Super-Resolution Quality Assessment: A Real-World Dataset, Subjective Studies, and an Objective Metric

Jiang, Qiuping; Liu, Zhentao; Gu, Ke; Shao, Feng; Zhang, Xinfeng; Liu, Hantao; Wang, Lin

doi:10.1109/tip.2022.3154588

Cited by 72 publications

(17 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To further investigate the statistical significance of the proposed TSNet, we perform the two-sample T-Test [14] on to compare the effectiveness in pairs. We conduct the test by randomly choose 20 images and calculate the SRCC result.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…Ma et al devised a NR-IQA method to regress the perceptual quality of SR images [9]. Jiang et al provided a new perceptive on SR metric by splitting the structural and textural information with Karhunen-Loéve Transformation [14]. Zhang et al integrated AdaBoost decision tree regression and ridge regression to predict the quality score [10].…”

Section: Image Quality Assessment For Image Super-resolutionmentioning

confidence: 99%

“…The pre-trained deep CNN holds effective capacity for structural feature exploration, but lacks to describe the textural degradation by the high-level feature representation [13]. Recently, KLTSRQA [14] utilizes Karhunen-Loéve Transform (KLT) to separate the structural and textural information and regresses the MOS. However, the hand-crafted extractors have no adaptive ability to learn the diverse SR features and limits the accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Textural-Perceptual Joint Learning for No-Reference Super-Resolution Image Quality Assessment

Liu¹,

Jia²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Image super-resolution (SR) has been widely investigated in recent years. However, it is challenging to fairly estimate the performances of various SR methods, as the lack of reliable and accurate criteria for perceptual quality. Existing SR image quality assessment (IQA) metrics usually concentrate on the specific kind of degradation without distinguishing the visual sensitive areas, which have no adaptive ability to describe the diverse SR degeneration situations. In this paper, we focus on the textural and structural degradation of image SR which acts as a critical role for visual perception, and design a dual stream network to jointly explore the textural and structural information for quality prediction, dubbed TSNet. By mimicking the human vision system (HVS) that pays more attention to the significant areas of the image, we develop the spatial attention mechanism to make the visual-sensitive areas more distinguishable, which improves the prediction accuracy. Feature normalization (F-Norm) is also developed to investigate the inherent spatial correlation of SR features and boost the network representation capacity. Experimental results show the proposed TSNet predicts the visual quality more accurate than the state-ofthe-art IQA methods, and demonstrates better consistency with the human's perspective. The source code will be made available at http://github.com/yuqing-liu-dut/NRIQA SR.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: Image Quality Assessment For Image Super-resolutionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Textural-Perceptual Joint Learning for No-Reference Super-Resolution Image Quality Assessment

Liu¹,

Jia²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Single image super-resolution. SISR has many research branches, such as the classical bicubic degradation SR (including PSNR-oriented and perception-driven [14,15]), real SR [16,17], reference-based SR [18,19] and so on. This paper mainly focuses on PSNR-oriented algorithms.…”

Section: Related Workmentioning

confidence: 99%

Learning Knowledge Representation with Meta Knowledge Distillation for Single Image Super-Resolution

Han¹,

Chen²,

Liu³

2022

Preprint

View full text Add to dashboard Cite

Although the deep CNN-based super-resolution methods have achieved outstanding performance, their memory cost and computational complexity severely limit their practical employment. Knowledge distillation (KD), which can efficiently transfer knowledge from a cumbersome network (teacher) to a compact network (student), has demonstrated its advantages in some computer vision applications. The representation of knowledge is vital for knowledge transferring and student learning, which is generally defined in hand-crafted manners or uses the intermediate features directly. In this paper, we propose a model-agnostic meta knowledge distillation method under the teacher-student architecture for the single image super-resolution task. It provides a more flexible and accurate way to help the teachers transmit knowledge in accordance with the abilities of students via knowledge representation networks (KRNets) with learnable parameters. In order to improve the perception ability of knowledge representation to students' requirements, we propose to solve the transformation process from intermediate outputs to transferred knowledge by employing the student features and the correlation between teacher and student in the KRNets. Specifically, the texture-aware dynamic kernels are generated and then extract texture features to be improved and the corresponding teacher guidance so as to decompose the distillation problem into texture-wise supervision for further promoting the recovery quality of high-frequency details. In addition, the KRNets are optimized in a meta-learning manner to ensure the knowledge transferring and the student learning are beneficial to improving the reconstructed quality of the student. Experiments conducted on various single image super-resolution datasets demonstrate that our proposed method outperforms existing defined knowledge representation related distillation methods, and can help super-resolution algorithms achieve better reconstruction quality without introducing any inference complexity.

show abstract

“…For test datasets with both datasets of paired images and datasets without reference images, we introduced multiple evaluation metrics (PSNR, SSIM [50], LOE [51], FSIM [52,53], NIQE [54], BRISQUE [55], ILNIQE [56,57]) for different datasets to ensure the objectivity and adequacy of the enhanced results. Among them, PSNR, SSIM, LOE and FSIM are evaluation metrics that require the use of reference images and can only be used on paired images.…”

Section: Evaluation Indexmentioning

confidence: 99%

Attention‐based multi‐channel feature fusion enhancement network to process low‐light images

Hua

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

In realistic low‐light environments, images captured by imaging devices often have problems such as low brightness and low contrast, serious loss of detail information, and a large amount of noise, posing major challenges to computer vision tasks. Low‐light image enhancement can effectively improve the overall quality of the image, which has important significance and application value. In this study, an attention‐based multi‐channel feature fusion enhancement network (M‐FFENet) is proposed to process low‐light images. In this network, a feature extraction model is first used to obtain the deep features of the downsampled low‐light images and fit them to an affine bilateral grid. Second, the addition of attention‐based residual dense blocks (ARDB) allows the network to focus on more details and spatial information. Meanwhile, all color channels are considered. The channel features and bilateral meshes are then linearly interpolated using the feature reconfiguration model (FRM) to obtain high‐quality features containing rich color and texture information. Next, the feature fusion module (FFM) is used to fuse features that contain different information. Enhancement model is used to further recover texture and detail in the image. Finally, the enhanced image is output. Numerous experimental results have shown that the method achieves better results in both quantitative and qualitative aspects compared to other methods.

show abstract

Single Image Super-Resolution Quality Assessment: A Real-World Dataset, Subjective Studies, and an Objective Metric

Cited by 72 publications

References 78 publications

Textural-Perceptual Joint Learning for No-Reference Super-Resolution Image Quality Assessment

Textural-Perceptual Joint Learning for No-Reference Super-Resolution Image Quality Assessment

Learning Knowledge Representation with Meta Knowledge Distillation for Single Image Super-Resolution

Attention‐based multi‐channel feature fusion enhancement network to process low‐light images

Contact Info

Product

Resources

About