A computational model for predicting local distortion visibility via convolutional neural network trainedon natural scenes

Alam, Mushfiqul; Patil, Pranita; Hagan, Martin T.; Chandler, Damon M.

doi:10.1109/icip.2015.7351550

Cited by 5 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the inference step does not require the knowledge of the error, and can produce an estimate of local masking for any input image. Interestingly (and perhaps surprisingly), we find that perceptual scaling learned from image quality scores can predict the detection thresholds in [9] with similar accuracy as the CNN-based regressor in [10], although our model is learned on other datasets with different contents and several kinds of visual impairements. This makes the proposed approach potentially more general than previous work.…”

Section: Introductionmentioning

confidence: 68%

“…We observe that these points correspond indeed to the outliers in the scatter plot. By removing these few very dark or very bright patches (28 patches out of 1080), we ob- 6.521 Teo & Heeger [22] 6.861 Chandler et al [1] 6.879 Optimized GC [10] 5.192 Alam et al CNN [10] 5.475…”

Section: Performancementioning

confidence: 99%

“…We compare in Table 1 the performance of our method (trained with three different IQA datasets) with other predictors of local visibility thresholds proposed in the literature. The majority of these methods use handcrafted models directly derived from psychophysical experiment, while [10] employs a convolutional neural network. The performance criterion is root-mean-squared error (RMSE) between the ground-truth and predicted thresholds.…”

Section: Performancementioning

confidence: 99%

“…Specifically, they use Gabor patterns as stimuli and natural images as masks to measure detection thresholds -the minimum magnitude of the stimulus making it distinguishable from the background mask -for small image patches. Later, this data has been used to train a convolutional neural network in order to predict the detection threshold of a patch from its input pixel values [10]. Conventional approaches to model distortion visibility strongly rely upon psychophysical experiments that are, in their nature, based on a simplification of real-world conditions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning Local Distortion Visibility from Image Quality

Kottayil

Cheng

Valenzise

et al. 2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Accurate prediction of local distortion visibility thresholds is critical in many image and video processing applications. Existing methods require an accurate modeling of the human visual system, and are derived through pshycophysical experiments with simple, artificial stimuli. These approaches, however, are difficult to generalize to natural images with complex types of distortion. In this paper, we explore a different perspective, and we investigate whether it is possible to learn local distortion visibility from image quality scores. We propose a convolutional neural network based optimization framework to infer local detection thresholds in a distorted image. Our model is trained on multiple quality datasets, and the results are correlated with empirical visibility thresholds collected on complex stimuli in a recent study. Our results are comparable to state-of-the-art mathematical models that were trained on phsycovisual data directly. This suggests that it is possible to predict psychophysical phenomena from visibility information embedded in image quality scores.

show abstract

Section: Introductionmentioning

confidence: 68%

Section: Performancementioning

confidence: 99%

Section: Performancementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Local Distortion Visibility from Image Quality

Kottayil

Cheng

Valenzise

et al. 2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…However, these earlier models were limited by the available computational power at their time, which required them to tailor their models to the processed stimuli or to limit the possible computations, for example, to entirely local normalization. Recently some more models were implemented to work on images (e.g., Alam, Patil, Hagan, & Chandler, 2015;Bradley, Abrams, & Geisler, 2014). These models usually do not cover the whole complexity, but simplify the normalization steps to reach a computationally more efficient model (Bradley et al, 2014) or are based on entirely different approaches like neural networks trained to predict the detectability of specific distortions (Alam et al, 2015).…”

Section: History and Classical Experiments In Spatial Visionmentioning

confidence: 99%

An image-computable psychophysical spatial vision model

Schütt

Wichmann

2017

Journal of Vision

View full text Add to dashboard Cite

A large part of classical visual psychophysics was concerned with the fundamental question of how pattern information is initially encoded in the human visual system. From these studies a relatively standard model of early spatial vision emerged, based on spatial frequency and orientation-specific channels followed by an accelerating nonlinearity and divisive normalization: contrast gain-control. Here we implement such a model in an image-computable way, allowing it to take arbitrary luminance images as input. Testing our implementation on classical psychophysical data, we find that it explains contrast detection data including the ModelFest data, contrast discrimination data, and oblique masking data, using a single set of parameters. Leveraging the advantage of an image-computable model, we test our model against a recent dataset using natural images as masks. We find that the model explains these data reasonably well, too. To explain data obtained at different presentation durations, our model requires different parameters to achieve an acceptable fit. In addition, we show that contrast gain-control with the fitted parameters results in a very sparse encoding of luminance information, in line with notions from efficient coding. Translating the standard early spatial vision model to be image-computable resulted in two further insights: First, the nonlinear processing requires a denser sampling of spatial frequency and orientation than optimal coding suggests. Second, the normalization needs to be fairly local in space to fit the data obtained with natural image masks. Finally, our image-computable model can serve as tool in future quantitative analyses: It allows optimized stimuli to be used to test the model and variants of it, with potential applications as an image-quality metric. In addition, it may serve as a building block for models of higher level processing.

show abstract

Near-threshold perceptual distortion prediction based on optimal structure classification

Liu

Allebach

2016

2016 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

A computational model for predicting local distortion visibility via convolutional neural network trainedon natural scenes

Cited by 5 publications

References 22 publications

Learning Local Distortion Visibility from Image Quality

Learning Local Distortion Visibility from Image Quality

An image-computable psychophysical spatial vision model

Near-threshold perceptual distortion prediction based on optimal structure classification

Contact Info

Product

Resources

About