Contrastive distortion‐level learning‐based no‐reference image‐quality assessment

Wei, Xuekai; Li, Jin; Zhou, Mingliang; Wang, Xianmin

doi:10.1002/int.22965

Cited by 11 publications

(3 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…to extract latent embedding features which are used to measure the difference (e.g., distortion type, distortion level, and content category) between the current image sample and the other samples in the training database. Recently, various distortion descriptors have been developed to extract global embedding features, such as distortion type-based discriminative learning [49], distortion level-based contrastive learning [83], and content categorybased similarity learning [59]. In general, these metric-based BIQAs treat training samples as mutual quality references and maximize the difference in quality features.…”

Section: Unsupervised Learning-based Biqasmentioning

confidence: 99%

Blind Multimodal Quality Assessment: A Brief Survey and A Case Study of Low-light Images

Wang¹,

Xu²,

Xu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Blind image quality assessment (BIQA) aims at automatically and accurately forecasting objective scores for visual signals, which has been widely used to monitor product and service quality in low-light applications, covering smartphone photography, video surveillance, autonomous driving, etc. Recent developments in this field are dominated by unimodal solutions inconsistent with human subjective rating patterns, where human visual perception is simultaneously reflected by multiple sensory information (e.g., sight and hearing). In this article, we present a unique blind multimodal quality assessment (BMQA) of low-light images from subjective evaluation to objective score. To investigate the multimodal mechanism, we first establish a multimodal low-light image quality (MLIQ) database with authentic low-light distortions, containing image and audio modality pairs. Further, we specially design the key modules of BMQA, considering multimodal quality representation, latent feature alignment and fusion, and hybrid self-supervised and supervised learning. Extensive experiments show that our BMQA yields state-of-the-art accuracy on the proposed MLIQ benchmark database. In particular, we also build an independent single-image modality Dark-4K database, which is used to verify its applicability and generalization performance in mainstream unimodal applications. Qualitative and quantitative results on Dark-4K show that BMQA achieves superior performance to existing BIQA approaches as long as a pre-trained quality semantic description model is provided. The proposed framework and two databases as well as the collected BIQA methods and evaluation metrics are made publicly available.

show abstract

Section: Unsupervised Learning-based Biqasmentioning

confidence: 99%

Blind Multimodal Quality Assessment: A Brief Survey and A Case Study of Low-light Images

Wang¹,

Xu²,

Xu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Deep learning-based approaches have also been proposed for NF-IQA [35][36][37][38][39][40][41][42]. Multi-scale architecture [38,39], transformer model [38,40,41], and contrastive learning [42,43] have recently been introduced.…”

Section: Plos Onementioning

confidence: 99%

GuaRD: Guaranteed robustness of image retrieval system under data distortion turbulence

Chung,

Lee,

Lee

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Image search systems could be endangered by adversarial attacks and data perturbations. The image retrieval system can be compromised either by distorting the query or hacking the ranking system. However, existing literature primarily discusses attack methods, whereas the research on countermeasures to defend against such adversarial attacks is rare. As a defense mechanism against the intrusions, quality assessment can complement existing image retrieval systems. “GuaRD” is proposed as an end-to-end framework that uses the quality metric as a weighted-regularization term. Proper utilization and balance of the two features could lead to reliable and robust ranking; the original image is assigned a higher rank while the distorted image is assigned a relatively lower rank. Meanwhile, the primary goal of the image retrieval system is to prioritize searching the relevant images. Therefore, the use of leveraged features should not compromise the accuracy of the system. To evaluate the generality of the framework, we conducted three experiments on two image quality assessment(IQA) benchmarks (Waterloo and PieAPP). For the first two tests, GuaRD achieved enhanced performance than the existing model: the mean reciprocal rank(mRR) value of the original image predictions increased by 61%, and the predictions for the distorted input query decreased by 18%. The third experiment was conducted to analyze the mean average precision (mAP) score of the system to verify the accuracy of the retrieval system. The results indicated little deviation in performance between the tested methods, and the score was not effected or slightly decreased by 0.9% after the GuaRD was applied. Therefore, GuaRD is a novel and robust framework that can act as a defense mechanism for data distortions.

show abstract

“…Meanwhile, conventional learning models including sparse coding [15], neighborhood regression [16], and random forest [17] also have a signifcant impact in this feld. In recent years, the SR capacity has been signifcantly improved by deep convolutional neural networks (CNN) [18][19][20][21][22][23] as in other information processing areas such as object detection [24], disease classifcation [25], segmentation [26], emotion and activity recognition [27,28], image generation [29], manipulation and deepfake detection [30,31], and quality enhancement and assessment [32][33][34][35][36]. However, most deep CNNbased SISR methods boost performance via deepening or widening networks, resulting in quite a few network parameters and high computational complexity.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Attention Feature Distillation Neural Network for Lightweight Single Image Super-Resolution

Zhang,

Lin,

Yang

et al. 2024

International Journal of Intelligent Systems

View full text Add to dashboard Cite

In recent years, remarkable performance improvements have been produced by deep convolutional neural networks (CNN) for single image super-resolution (SISR). Nevertheless, a high proportion of CNN-based SISR models are with quite a few network parameters and high computational complexity for deep or wide architectures. How to more fully utilize deep features to make a balance between model complexity and reconstruction performance is one of the main challenges in this field. To address this problem, on the basis of the well-known information multi-distillation model, a multi-attention feature distillation network termed as MAFDN is developed for lightweight and accurate SISR. Specifically, an effective multi-attention feature distillation block (MAFDB) is designed and used as the basic feature extraction unit in MAFDN. With the help of multi-attention layers including pixel attention, spatial attention, and channel attention, MAFDB uses multiple information distillation branches to learn more discriminative and representative features. Furthermore, MAFDB introduces the depthwise over-parameterized convolutional layer (DO-Conv)-based residual block (OPCRB) to enhance its ability without incurring any parameter and computation increase in the inference stage. The results on commonly used datasets demonstrate that our MAFDN outperforms existing representative lightweight SISR models when taking both reconstruction performance and model complexity into consideration. For example, for ×4 SR on Set5, MAFDN (597K/33.79G) obtains 0.21 dB/0.0037 and 0.10 dB/0.0015 PSNR/SSIM gains over the attention-based SR model AFAN (692K/50.90G) and the feature distillation-based SR model DDistill-SR (675K/32.83G), respectively.

show abstract

Contrastive distortion‐level learning‐based no‐reference image‐quality assessment

Cited by 11 publications

References 54 publications

Blind Multimodal Quality Assessment: A Brief Survey and A Case Study of Low-light Images

Blind Multimodal Quality Assessment: A Brief Survey and A Case Study of Low-light Images

GuaRD: Guaranteed robustness of image retrieval system under data distortion turbulence

A Multi-Attention Feature Distillation Neural Network for Lightweight Single Image Super-Resolution

Contact Info

Product

Resources

About