End-to-End Blind Image Quality Prediction With Cascaded Deep Neural Network

Wu, Jinjian; Ma, Jianpeng; Liang, Fuhu; Dong, Weisheng; Shi, Guangming; Wang, Lin

doi:10.1109/tip.2020.3002478

Cited by 159 publications

(72 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such approach cannot be really modelled in an end-to-end manner as the quality of image patches often differ from the whole image. Later, end-to-end models have been proposed that can predict image quality directly [11]- [14]. Ma et al [11] proposed MEON consisting of two sub-networks for distortion identification and quality prediction that share early layers.…”

Section: Related Workmentioning

confidence: 99%

“…Yang et al [13] proposed a multi-task network (SGDNet) to predict image saliency and quality jointly, and image saliency can also serve as input to generate more perceptually consistent features for IQA. A cascaded architecture (CaHDC) was proposed in [14] that can represent the hierarchical perception mechanism in HVS, and then combine the features extracted at different scales by a simple side pooling net. The above NR-IQA models both assume the input images have same resolution, so that image rescaling is not required.…”

Section: Related Workmentioning

confidence: 99%

“…The above NR-IQA models both assume the input images have same resolution, so that image rescaling is not required. Nevertheless, these models in [12]- [14] are built by convolutional blocks, whose parameters are actually independent of input shape. The models can be adjusted to accept arbitrary image resolutions by global pooling before the multi-layer perceptron.…”

Section: Related Workmentioning

confidence: 99%

“…It has also attracted wide research interest in no-reference (NR) visual qual-ity assessment, especially image quality assessment (IQA). Most NR-IQA models driven by deep learning follows an approach employing CNN as a feature extractor and single-or multi-layer perceptron for quality prediction, e.g., see [10]- [14]. These NR-IQA models often accept constant input image resolutions by either image rescaling or patching, due to that the shape of the single-or multi-layer perceptron need to be predefined and most existing CNN architectures have fixed input sizes.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Long Short-term Convolutional Transformer for No-Reference Video Quality Assessment

You

2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

No-reference video quality assessment has not been widely benefited from deep learning, mainly due to the complexity, diversity and particularity of modelling spatial and temporal characteristics in quality assessment scenario. Image quality assessment (IQA) performed on video frames plays a key role in NR-VQA. A perceptual hierarchical network (PHIQNet) with an integrated aention module is first proposed that can appropriately simulate the visual mechanisms of contrast sensitivity and selective aention in IQA. Subsequently, perceptual quality features of video frames derived from PHIQNet are fed into a long shortterm convolutional Transformer (LSCT) architecture to predict the perceived video quality. LSCT consists of CNN formulating quality features in video frames within short-term units that are then fed into Transformer to capture the long-range dependence and attention allocation over temporal units. Such architecture is in line with the intrinsic properties of VQA. Experimental results on publicly available video quality databases have demonstrated that the LSCT architecture based on PHIQNet significantly outperforms state-of-the-art video quality models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Long Short-term Convolutional Transformer for No-Reference Video Quality Assessment

You

2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…As the number of images in the newly released IQA datasets becomes larger, deep learning came into play and began to dominate the field of BIQA. Many strategies were proposed to compensate for the lack of humanlabeled data, including patchwise training [13], [30], transfer learning [31], and quality-aware pre-training [14], [32], [33], [34], [35]. To confront the synthetic-to-real challenge (and vice versa), Zhang et al [19], [20] proposed a computational method of training BIQA models on multiple datasets.…”

Section: Biqa Modelsmentioning

confidence: 99%

Learning to Rank for Blind Image Quality Assessment

Zheng

Zhang

Cai

2020

Proceedings of the 4th International Conference on Computer Science and Application Engineering

View full text Add to dashboard Cite

The explosive growth of image data facilitates the fast development of image processing and computer vision methods for emerging visual applications, meanwhile introducing novel distortions to the processed images. This poses a grand challenge to existing blind image quality assessment (BIQA) models, failing to continually adapt to such subpopulation shift. Recent work suggests training BIQA methods on the combination of all available human-rated IQA datasets. However, this type of approach is not scalable to a large number of datasets, and is cumbersome to incorporate a newly created dataset as well. In this paper, we formulate continual learning for BIQA, where a model learns continually from a stream of IQA datasets, building on what was learned from previously seen data. We first identify five desiderata in the new setting with a measure to quantify the plasticity-stability trade-off. We then propose a simple yet effective method for learning BIQA models continually. Specifically, based on a shared backbone network, we add a prediction head for a new dataset, and enforce a regularizer to allow all prediction heads to evolve with new data while being resistant to catastrophic forgetting of old data. We compute the quality score by an adaptive weighted summation of estimates from all prediction heads. Extensive experiments demonstrate the promise of the proposed continual learning method in comparison to standard training techniques for BIQA.

show abstract