Multiview Contrastive Learning for Completely Blind Video Quality Assessment of User Generated Content

Mitra, Shankhanil; Soundararajan, Rajiv

doi:10.1145/3503161.3548064

Cited by 5 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsupervised VQA. VQA methods such as STEM (Kancharla and Channappayya 2022), VISION (Mitra and Soundararajan 2022), and NVQE (Liao et al 2022) do not require any human labelled videos in their design and give reasonable quality estimates for UGC videos. Nevertheless, their performance with respect to the methods trained with human opinion scores is under par.…”

Section: Related Workmentioning

confidence: 99%

“…To capture quality-aware representations, we choose contrastive pairs of video clips from synthetically distorted UGC videos having similar content but different levels and types of distortions. We synthetically distort UGC videos similar to VISION (Mitra and Soundararajan 2022) to model mixed camera captured and synthetic distortions from which…”

Section: Spatio-temporal Vq Representation Learningmentioning

confidence: 99%

“…Among recent deep VQA models, we compare with the state-of-the-art FAST-VQA model (Wu et al 2022) and the popular VSFA (Li, Jiang, and Jiang 2019) and TCSVT-BVQA (Li et al 2022) methods. Finally, we also compare with recent self-supervised feature learning models such as VISION (Mitra and Soundararajan 2022) and CONVIQT (Madhusudana et al 2022). The comparison with the supervised pre-trained FAST-VQA model is particularly interesting since the experiment reveals how our self-supervised ST-VQRL backbone of SSL-VQA − enables learning with limited labelled data.…”

Section: Experimental Setting 1 Analysismentioning

confidence: 99%

“…With this motivation, we approach the problem using a combination of contrastive self-supervised pretraining followed by semisupervised finetuning. A few self-supervised contrastive learning based methods have been designed for NR VQA recently (Mitra and Soundararajan 2022;Madhusudana et al 2022;Chen et al 2022) to learn rich video quality features. However, none of these methods yet exploit the performance benefit offered by the attention mechanism in transformer based models.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Knowledge Guided Semi-supervised Learning for Quality Assessment of User Generated Videos

Mitra,

Soundararajan

2024

AAAI

View full text Add to dashboard Cite

Perceptual quality assessment of user generated content (UGC) videos is challenging due to the requirement of large scale human annotated videos for training. In this work, we address this challenge by first designing a self-supervised Spatio-Temporal Visual Quality Representation Learning (ST-VQRL) framework to generate robust quality aware features for videos. Then, we propose a dual-model based Semi Supervised Learning (SSL) method specifically designed for the Video Quality Assessment (SSL-VQA) task, through a novel knowledge transfer of quality predictions between the two models. Our SSL-VQA method uses the ST-VQRL backbone to produce robust performances across various VQA datasets including cross-database settings, despite being learned with limited human annotated videos. Our model improves the state-of-the-art performance when trained only with limited data by around 10%, and by around 15% when unlabelled data is also used in SSL. Source codes and checkpoints are available at https://github.com/Shankhanil006/SSL-VQA.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Spatio-temporal Vq Representation Learningmentioning

confidence: 99%

Section: Experimental Setting 1 Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Knowledge Guided Semi-supervised Learning for Quality Assessment of User Generated Videos

Mitra,

Soundararajan

2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Similar to FID, FVD also demonstrates a weak correlation with human visual perception. As related research, user-generated content (UGC) VQA models [57], such as SimpleVQA [58], FastVQA [59], DOVER [60], OV-PSNR [61], SSL [62], etc., have attempted to utilize action recognition network (e.g. SlowFast [20], Video Swin Transformer [63]) to represent the temporal quality feature.…”

Section: B Quality Metrics For Aigc Videosmentioning

confidence: 99%

Surveillance Video Quality Assessment Based on Quality Related Retraining

Zhang

Lü

Sun

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Surveillance videos have been widely used in many visionbased systems, supporting intelligent tasks such as object detection and tracking. However, the quality of surveillance videos suffers from poor weather conditions and inevitable compression error, which may have a negative influence on the performance of such tasks. Therefore, accurately distinguishing distortions and predicting severity levels are crucial. In this paper, we propose a quality related retraining framework as well as a no-reference (NR) multi-task video quality assessment (VQA) model to tackle the challenge of surveillance videos quality assessment. The quality related retraining framework operates on a synthetic VQA database. The proposed NR VQA method utilizes both spatial and temporal information by using ResNet50 and SlowFast. Then multiple distortion detection heads are applied to predict the severity levels for corresponding distortions. The experimental results show that the proposed method gains competitive performance on the Video Surveillance Quality Assessment Dataset (VSQuAD). The ablation study further confirms the contributions of the quality related retraining framework, spatial information, and temporal information.

show abstract

Perceptual video quality assessment: a survey

Min,

Duan,

Sun

et al. 2024

Sci. China Inf. Sci.

View full text Add to dashboard Cite

Perceptual video quality assessment plays a vital role in the field of video processing due to the existence of quality degradations introduced in various stages of video signal acquisition, compression, transmission and display. With the advancement of Internet communication and cloud service technology, video content and traffic are growing exponentially, which further emphasizes the requirement for accurate and rapid assessment of video quality. Therefore, numerous subjective and objective video quality assessment studies have been conducted over the past two decades for both generic videos and specific videos such as streaming, user-generated content, 3D, virtual and augmented reality, high dynamic range, high frame rate, audio-visual, etc. This survey provides an up-to-date and comprehensive review of these video quality assessment studies. Specifically, we first review the subjective video quality assessment methodologies and databases, which are necessary for validating the performance of video quality metrics. Second, the objective video quality assessment measures for general purposes are categorized and surveyed according to the methodologies utilized in the quality measures. Third, we overview the objective video quality assessment measures for specific applications and emerging topics. Finally, the performance of the state-of-the-art video quality assessment measures is compared and analyzed. This survey provides a systematic overview of both classical works and recent progress in the realm of video quality assessment, which can help other researchers quickly access the field and conduct relevant research.

show abstract

Multiview Contrastive Learning for Completely Blind Video Quality Assessment of User Generated Content

Cited by 5 publications

References 30 publications

Knowledge Guided Semi-supervised Learning for Quality Assessment of User Generated Videos

Knowledge Guided Semi-supervised Learning for Quality Assessment of User Generated Videos

Surveillance Video Quality Assessment Based on Quality Related Retraining

Perceptual video quality assessment: a survey

Contact Info

Product

Resources

About