Subjective Assessment of Global Picture-Wise Just Noticeable Difference

Jenadeleh, Mohsen; Chen, Guangan; Reips, Ulf‐Dietrich; Hamzaoui, Raouf; Saupe, Dietmar

doi:10.1109/icmew46912.2020.9106058

Cited by 8 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our pilot study [29], we introduced and evaluated two new adjustment methods for subjective PJND assessment, where a subject interactively selects the distortion level at the PJND location by a slider or keystrokes. For both methods, a reference image and its distorted version are compared using the flicker test in which the displayed images alternate at a frequency of 8 Hz, as in the JPEG-XS standard [1].…”

Section: Pilot Study For Pjnd Assessment Methodsmentioning

confidence: 99%

Large-Scale Crowdsourced Subjective Assessment of Picturewise Just Noticeable Difference

Chen

Jenadeleh

Hosu

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

The picturewise just noticeable difference (PJND) for a given image, compression scheme, and subject is the smallest distortion level that the subject can perceive when the image is compressed with this compression scheme. The PJND can be used to determine the compression level at which a given proportion of the population does not notice any distortion in the compressed image. To obtain accurate and diverse results, the PJND must be determined for a large number of subjects and images. This is particularly important when experimental PJND data are used to train deep learning models that can predict a probability distribution model of the PJND for a new image. To date, such subjective studies have been carried out in laboratory environments. However, the number of participants and images in all existing PJND studies is very small because of the challenges involved in setting up laboratory experiments. To address this limitation, we develop a framework to conduct PJND assessments via crowdsourcing. We use a new technique based on slider adjustment and a flicker test to determine the PJND. A pilot study demonstrated that our technique could decrease the study duration by 50% and double the perceptual sensitivity compared to the standard binary search approach that successively compares a test image side by side with its reference image. Our framework includes a robust and systematic scheme to ensure the reliability of the crowdsourced results. Using 1,008 source images and distorted versions obtained with JPEG and BPG compression, we apply our crowdsourcing framework to build the largest PJND dataset, KonJND-1k (Konstanz just noticeable difference 1k dataset). A total of 503 workers participated in the study, yielding 61,030 PJND samples that resulted in an average of 42 samples per source image. The KonJND-1k dataset is available at http://database.mmsp-kn.de/konjnd-1kdatabase.html Index Terms-Just noticeable difference (JND), satisfied user ratio (SUR), crowdsourcing, flicker test, JPEG, BPG, dataset I. INTRODUCTIONI Mage compression is essential to meet constraints on transmission bandwidth and storage. With increasing compression levels, an increasing number of users can perceive distortion in the compressed version of an original image. The smallest distortion level that a user can perceive when an

show abstract

Section: Pilot Study For Pjnd Assessment Methodsmentioning

confidence: 99%

Large-Scale Crowdsourced Subjective Assessment of Picturewise Just Noticeable Difference

Chen

Jenadeleh

Hosu

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In our pre-study [52], presented at the ICME 2020 Workshop on Data-driven Just Noticeable Difference for Multimedia Communication, we have provided the first experiment to compare the performance of the flicker test with conventional side-by-side comparisons. The purpose of the tests was to assess the JND for JPEG image compression.…”

Section: B Boosting By the Image Flicker Methodsmentioning

confidence: 99%

Subjective Image Quality Assessment With Boosted Triplet Comparisons

Men¹,

Jenadeleh

Saupe

2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

In subjective full-reference image quality assessment, a reference image is distorted at increasing distortion levels. The differences between perceptual image qualities of the reference image and its distorted versions are evaluated, often using degradation category ratings (DCR). However, the DCR has been criticized since differences between rating categories on this ordinal scale might not be perceptually equidistant, and observers may have different understandings of the categories. Pair comparisons (PC) of distorted images, followed by Thurstonian reconstruction of scale values, overcomes these problems. In addition, PC is more sensitive than DCR, and it can provide scale values in fractional, just noticeable difference (JND) units that express a precise perceptional interpretation. Still, the comparison of images of nearly the same quality can be difficult. We introduce boosting techniques embedded in more general triplet comparisons (TC) that increase the sensitivity even more. Boosting amplifies the artefacts of distorted images, enlarges their visual representation by zooming, increases the visibility of the distortions by a flickering effect, or combines some of the above. Experimental results show the effectiveness of boosted TC for seven types of distortion (color diffusion, jitter, high sharpen, JPEG 2000 compression, lens blur, motion blur, multiplicative noise). For our study, we crowdsourced over 1.7 million responses to triplet questions. We give a detailed analysis of the data in terms of scale reconstructions, accuracy, detection rates, and sensitivity gain. Generally, boosting increases the discriminatory power and allows to reduce the number of subjective ratings without sacrificing the accuracy of the resulting relative image quality values. Our technique paves the way to fine-grained image quality datasets, allowing for more distortion levels, yet with high-quality subjective annotations. We also provide the details for Thurstonian scale reconstruction from TC and our annotated dataset, KonFiG-IQA, containing 10 source images, processed using 7 distortion types at 12 or even 30 levels, uniformly spaced over a span of 3 JND units.

show abstract

“…The most commonly used presentations include 1) displaying a reference image and a distorted image sequentially [8], [12], 2) displaying a reference image and a distorted image side-by-side [9], [10], [13], and 3) a sensitive method known as the flicker test, as adopted by an ISO/IEC standard [14]. In addition, these methods can be categorized in terms of search strategies for PJND identification, including standard binary search [13], relaxed binary search [10], and slider-based search strategy [15].…”

Section: Introductionmentioning

confidence: 99%

Localization of Just Noticeable Difference for Image Compression

Chen

Wiedemann

Saupe

2023

2023 15th International Conference on Quality of Multimedia Experience (QoMEX)

Self Cite

View full text Add to dashboard Cite

The just noticeable difference (JND) is the minimal difference between stimuli that can be detected by a person. The picture-wise just noticeable difference (PJND) for a given reference image and a compression algorithm represents the minimal level of compression that causes noticeable differences in the reconstruction. These differences can only be observed in some specific regions within the image, dubbed as JND-critical regions. Identifying these regions can improve the development of image compression algorithms. Due to the fact that visual perception varies among individuals, determining the PJND values and JND-critical regions for a target population of consumers requires subjective assessment experiments involving a sufficiently large number of observers. In this paper, we propose a novel framework for conducting such experiments using crowdsourcing. By applying this framework, we created a novel PJND dataset, KonJND++, consisting of 300 source images, compressed versions thereof under JPEG or BPG compression, and an average of 43 ratings of PJND and 129 self-reported locations of JND-critical regions for each source image. Our experiments demonstrate the effectiveness and reliability of our proposed framework, which is easy to be adapted for collecting a large-scale dataset. The source code and dataset are available at https://github.com/angchen-dev/LocJND.

show abstract

Subjective Assessment of Global Picture-Wise Just Noticeable Difference

Cited by 8 publications

References 12 publications

Large-Scale Crowdsourced Subjective Assessment of Picturewise Just Noticeable Difference

Large-Scale Crowdsourced Subjective Assessment of Picturewise Just Noticeable Difference

Subjective Image Quality Assessment With Boosted Triplet Comparisons

Localization of Just Noticeable Difference for Image Compression

Contact Info

Product

Resources

About