Analyzing the Variability of Subjective Image Quality Ratings for Different Distortions

Abstract

When it comes to evaluating the quality of images, individual observers show different opinions depending on the type of distortion affecting the quality of the image. While in the opinion of one observer a distortion could have a dramatic influence on the quality of the image, another observer could see the same distortion as not having an important effect on the quality of the same image. Using a subjective experiment, we aim to identify the distortions which show the largest variability among observers. For this, 22 observers evaluated the quality of 10 reference images and the 630 test images created from them (21 distortions at three levels). Our results show that the highest variability in subjective scores is linked to distortions like saturation, contrast, sharpness, quantization, some types of added noise, and radial lens distortion.

Individual Contrast Preferences in Natural Images

“…A 3-AFC procedure is more efficient, stable, and precise compared to 2-AFC [61][62][63]. The algorithm used to change the contrast in the image was selected based on the findings of [1,15] and was adopted from the Kadid10K dataset [7]. (The contrast-changing algorithm MATLAB source code can be downloaded from the KADID-10k IQA database webpage:http://database.mmsp-kn.de/kadid-10k-database.html (accessed on 1 November 2021).…”

“…Personalized approaches are gaining increasing popularity not only in the field of Image Quality Assessment (IQA) [1] but also in various other domains [2]. For years, subjective experiments have delivered results in the form of Mean Opinion Scores (MOSs).…”

“…In this work, we focus on contrast, a specific image attribute that not only influences the perception of image quality [14] but also causes one of the largest variabilities among observers when used as a type of distortion in testing image quality preferences [1]. Exploring this particular attribute, our objective is to identify distinctive observer preferences in order to use this information to improve personalized image quality in the future.…”

“…In this paper, we evaluate individual preferences at different contrast levels for a wide range of images that all can be classified as images taken from natural landscapes (Table 1). This attribute was chosen among others, such as lightness, sharpness, colorfulness, etc., because it leads to one of the largest variabilities between observer opinions when evaluating image quality [1]. Cherepkova et al [15] show that observers have distinguishable patterns with regard to contrast distortion when judging image quality, where some observers rate images with higher contrast to have higher quality and vice versa.…”