Visual Attention in Quality Assessment

Abstract

P erceptual quality metrics are widely deployed in image and video processing systems. These metrics aim to emulate the integral mechanisms of the human visual system (HVS) to correlate well with visual perception of quality. One integral property of the HVS is, however, often neglected: visual attention (VA) [1]. The essential mechanisms associated with VA consist mainly of higher cognitive processing, deployed to reduce the complexity of scene analysis. For this purpose, a subset of the visual information is selected by shifting the focus of attention across the visual scene to the most relevant objects. By neglecting VA, perceptual quality models inherently assume that all objects draw the attention of the viewer to the same degree. This applies to both the natural scene content as well as possibly induced distortions. However, suprathreshold distortions can be a strong attractor of VA and as a result, have a severe impact on the perceived quality. Identifying the perceptual influence of distortions relative to the natural content can thus be expected to enhance the prediction performance of perceptual quality metrics. The potential benefit of integrating VA information into image and video quality models has recently been recognized by a number of research groups [2]- [20]. The conclusions drawn from these works are somewhat controversial and give rise to many open questions. The goals of this article are therefore to shed some light onto this immature research field and to provide guidance for further advances. Toward these goals, we first discuss VA concepts that are relevant in the context of quality perception. We then review recent advances in research on integrating VA into quality assessment and [ Theory, advances, and challenges ]

Compressed-domain visual saliency models: a comparative study

“…Eye-tracking data is the most typical psychophysical ground truth for visual saliency models [18]. To evaluate saliency models, each model's saliency map is compared with recorded gaze locations of the subjects.…”

Integrating Visual Saliency Information into Objective Quality Assessment of Tone-Mapped Images

“…Recently, researchers have gradually found that VS is closely relevant to the image's perceptual quality. Various methods have been attempted to integrate VS information into IQA metrics [4]. These methods demonstrate that a VS-weighted pooling strategy could perform better than the simple "mean" scheme.…”

“…Consequently, in this paper, we attempt to extent TMQI by proposing a novel VS-based pooling strategy. Specifically, in our method, we use VS as a weight map, which is inspired by its successful use on IQA [4,5]. The experimental results demonstrate that the VS-based pooling strategy can achieve significantly better performance than the simple "mean" that used in TMQI.…”

How Does Image Content Affect the Added Value of Visual Attention in Objective Image Quality Assessment?

“…A saliency map (SM) representative for visual attention is derived from gaze patterns recorded from eye-tracking (see [1], [4] and [6]). It is constructed adding to each fixation location a Gaussian patch, which approximates the size of the fovea in the human eye (about 2 of visual angle, corresponding to a σ of 45 pixels for the Gaussian for all three experiments).…”