Estimation of perceptual scales using ordinal embedding

Haghiri, Siavash; Wichmann, Felix A.; Luxburg, Ulrike von

doi:10.1167/jov.20.9.14

Cited by 13 publications

(26 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The oldest and most frequently used scaling algorithm for more than one dimension is (nonmetric) multidimensional scaling (MDS) ( Shepard, 1962 ; Kruskal, 1964a , 1964b )), which was recently accompanied by ordinal embedding methods from machine learning ( Roads & Mozer, 2019 ; Haghiri, Wichmann, & von Luxburg, 2020 ). In contrast with other scaling approaches, such as the popular maximum-likelihood difference scaling (MLDS) ( Knoblauch & Maloney, 2012 ), MDS and ordinal embedding can estimate multiple perceived dimensions.…”

Section: Introductionmentioning

confidence: 99%

Estimating the perceived dimension of psychophysical stimuli using triplet accuracy and hypothesis testing

2022

Self Cite

View full text Add to dashboard Cite

Vision researchers are interested in mapping complex physical stimuli to perceptual dimensions. Such a mapping can be constructed using multidimensional psychophysical scaling or ordinal embedding methods. Both methods infer coordinates that agree as much as possible with the observer’s judgments so that perceived similarity corresponds with distance in the inferred space. However, a fundamental problem of all methods that construct scalings in multiple dimensions is that the inferred representation can only reflect perception if the scale has the correct dimension. Here we propose a statistical procedure to overcome this limitation. The critical elements of our procedure are i) measuring the scale’s quality by the number of correctly predicted triplets and ii) performing a statistical test to assess if adding another dimension to the scale improves triplet accuracy significantly. We validate our procedure through extensive simulations. In addition, we study the properties and limitations of our procedure using “real” data from various behavioral datasets from psychophysical experiments. We conclude that our procedure can reliably identify (a lower bound on) the number of perceptual dimensions for a given dataset.

show abstract

Section: Introductionmentioning

confidence: 99%

Estimating the perceived dimension of psychophysical stimuli using triplet accuracy and hypothesis testing

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…The rapid increase in number of trials for this type of scaling procedure (Maximum Likelihood Difference Scaling (MLDS), MLCM) renders the procedures less attractive from a pragmatic point of view and might sometimes outweigh the theoretical benefits discussed above. Alternative strategies such as subsampling ( Knoblauch & Maloney, 2012 ; Abbatecola et al, 2021 ), having a reduced number of stimuli per dimension ( Sun et al, 2021 ), or the use of so-called embedded methods from the machine learning community (see Haghiri et al, 2020 , for example) are currently being explored and might allow more efficient ways of perceptual scale measurements in the future.…”

Section: Discussionmentioning

confidence: 99%

Conjoint measurement of perceived transparency and perceived contrast in variegated checkerboards

Aguilar

Maertens

2022

Journal of Vision

View full text Add to dashboard Cite

One fundamental question in vision research is how the retinal input is segmented into perceptually relevant variables. A striking example of this segmentation process is transparency perception, in which luminance information in one location contributes to two perceptual variables: the properties of the transparent medium itself and of what is being seen in the background. Previous work by Robilotto et al. (2002 , 2004) suggested that perceived transparency is closely related to perceived contrast, but how these two relate to retinal luminance has not been established. Here we studied the relationship between perceived transparency, perceived contrast, and image luminance using maximum likelihood conjoint measurement (MLCM). Stimuli were rendered images of variegated checkerboards that were composed of multiple reflectances and partially covered by a transparent overlay. We systematically varied the transmittance and reflectance of the transparent medium and measured perceptual scales of perceived transparency. We also measured scales of perceived contrast using cut-outs of the transparency stimuli that did not contain any geometrical cues to transparency. Perceptual scales for perceived transparency and contrast followed a remarkably similar pattern across observers. We tested the empirically observed scales against predictions from various contrast metrics and found that perceived transparency and perceived contrast were equally well predicted by a metric based on the logarithm of Michelson or Whittle contrast. We conclude that judgments of perceived transparency and perceived contrast are likely to be supported by a common mechanism, which can be computationally captured as a logarithmic contrast.

show abstract

“…There are several other methods for scale reconstruction from triplet comparisons, some of which have recently originated from the machine learning community [16]. Usually, these methods are for multi-dimensional scaling.…”

Section: Reconstruction Of Scale Values From Triplet Comparisonsmentioning

confidence: 99%

“…The most common and recommended multivariate outlier detection method in this spirit is a fast version of the minimum covariance determinant (Fast-MCD) approach [69]. However, the MCD method operates with Mahalanobis distances that are not suitable in our case since the multivariate data are not only vectors of ternary decisions rather than from a real Euclidean vector space, but also of variable dimension (16)(17)(18)(19).…”

Section: Robust Hit-level Outlier Removalmentioning

confidence: 99%

See 1 more Smart Citation

Subjective Image Quality Assessment With Boosted Triplet Comparisons

Men¹,

Jenadeleh

Saupe

2021

IEEE Access

View full text Add to dashboard Cite

In subjective full-reference image quality assessment, a reference image is distorted at increasing distortion levels. The differences between perceptual image qualities of the reference image and its distorted versions are evaluated, often using degradation category ratings (DCR). However, the DCR has been criticized since differences between rating categories on this ordinal scale might not be perceptually equidistant, and observers may have different understandings of the categories. Pair comparisons (PC) of distorted images, followed by Thurstonian reconstruction of scale values, overcomes these problems. In addition, PC is more sensitive than DCR, and it can provide scale values in fractional, just noticeable difference (JND) units that express a precise perceptional interpretation. Still, the comparison of images of nearly the same quality can be difficult. We introduce boosting techniques embedded in more general triplet comparisons (TC) that increase the sensitivity even more. Boosting amplifies the artefacts of distorted images, enlarges their visual representation by zooming, increases the visibility of the distortions by a flickering effect, or combines some of the above. Experimental results show the effectiveness of boosted TC for seven types of distortion (color diffusion, jitter, high sharpen, JPEG 2000 compression, lens blur, motion blur, multiplicative noise). For our study, we crowdsourced over 1.7 million responses to triplet questions. We give a detailed analysis of the data in terms of scale reconstructions, accuracy, detection rates, and sensitivity gain. Generally, boosting increases the discriminatory power and allows to reduce the number of subjective ratings without sacrificing the accuracy of the resulting relative image quality values. Our technique paves the way to fine-grained image quality datasets, allowing for more distortion levels, yet with high-quality subjective annotations. We also provide the details for Thurstonian scale reconstruction from TC and our annotated dataset, KonFiG-IQA, containing 10 source images, processed using 7 distortion types at 12 or even 30 levels, uniformly spaced over a span of 3 JND units.

show abstract

Estimation of perceptual scales using ordinal embedding

Cited by 13 publications

References 57 publications

Estimating the perceived dimension of psychophysical stimuli using triplet accuracy and hypothesis testing

Estimating the perceived dimension of psychophysical stimuli using triplet accuracy and hypothesis testing

Conjoint measurement of perceived transparency and perceived contrast in variegated checkerboards

Subjective Image Quality Assessment With Boosted Triplet Comparisons

Contact Info

Product

Resources

About