From Sparse Coding Significance to Perceptual Quality: A New Approach for Image Quality Assessment

Abstract

An increasing number of image processing applications require an automated quality prediction of the visual content as perceived by humans. Since, sparse coding is suggested to be an underlying strategy of the brain's neural system, it would be logical to assume that specific tasks like quality assessment also attempt to adhere to this strategy. However, existing perceptual quality predictors, often mimicking the different stages of the human visual system and deploying machine learning strategies, such as neural networks, rarely integrate the concept of sparse coding in their design. In this paper, we first investigate the validity of such assumption by performing an empirical analysis on the relation between the structural information of the scene-captured via sparseness significance-and perceptual quality. Subsequently, we propose a new approach to integrate the significance of sparse coding features in the future imagequality measure (IQM) designs. We utilize the Fourier transform as a case study, which leads to a new IQM called sparseness significance ranking measure (SSRM). This measure essentially deploys a Fourier basis for sparse coding, a ranking mechanism based upon the amplitudes of the sparse coefficients and subsequently a complex correlation metric that assesses the correspondence between the ranked coefficient amplitude profiles of the reference and the distorted image. Moreover, we introduce a new methodology, namely separation ratio analysis, to assess the prediction quality of individual features or quality predictors given a target perceptual quality. The quality predictions by the proposed SSRM show excellent compatibility with perceptual quality scores. A set of routine benchmarking experiments utilizing the LIVE and CSIQ, IVC and TID2008 databases indicates a highly competitive performance with state of the art IQMs. Moreover, it delivers this performance at a low computational cost.

Blind Tone-Mapped Image Quality Assessment Based on Regional Sparse Response and Aesthetics

“…From the perspective of neurophysiology [ 27 ], when visual neurons receive the external stimuli, the information carried by the stimulus can be correctly perceived, while sparse representation is exactly consistent with the perceptual process of the visual signal. Moreover, according to the previous studies about visual signal processing, it has been proven that sparse representation can effectively match the visual perception characteristics of mammalian organism and describe the image signals with their sparsity and redundancy [ 28 , 29 , 30 ]. Therefore, sparse representation is used to identify the specific distortion of TMI in this study, i.e., regional and global detail loss.…”

“… Three TMIs from the ESPL-LIVE HDR database [ 29 ] and the corresponding histograms of reconstructed images with SC coeff-l . ( a ) TMI generated by DurandTMO (mean opinion score (MOS) = 27.10); ( b ) TMI generated by FattalTMO (MOS = 47.99); ( c ) TMI generated by ReinhardTMO (MOS = 60.30) and ( d – f ) corresponding histograms of reconstructed images in the first row.…”

JPEG Pleno: Providing representation interoperability for holographic applications and devices

“… recently presented a versatile similarity measure that attempts to resolve this caveat by proposing a multifactor measure (considering the factors accounting for amplitude, phase and sign differences) while ’annealing’ incidental phase wraps. Moreover, also a sparseness significance ranking measure (SSRM) was proposed that is based on sparse coding and a ranking system for the magnitudes of the spatial frequency coefficients. This technique was shown to also be more effective than PSNR or SSIM on holographic data .…”

A Roadmap Towards Holographic Television From a Signal Processing Perspective

“…visual masking, contrast sensitivity function), their importance for sparse coding, namely sparseness significance was considered for differentiation. We proposed a new characterization of structural information in the Fourier domain based on this idea [7]. After evaluating the behaviour of Fourier components of the distorted images in complex plane utilizing the notion of distortion vectors, we designed a novel perceptual quality predictor.…”