Perception of video quality at a local spatio-temporal horizon

Abstract

This paper contains the research proposal of Andréas Pastor that was presented at the MMSys 2022 doctoral symposium. Encoding video for streaming on Internet has become a major topic to reduce the consumption of bandwidth and latency. At the same time, the human perception of distortions has been explored in multiple research projects, especially for distortions generated by Coder-DECoder (CODEC) algorithms. These algorithms operate in a rate-distortion optimization paradigm to efficiently compress video content. This optimization can be driven by metrics that are most of the time not based on the human perception, and more importantly, not tuned to reflect the local perception of distortions by human eyes.In this doctoral study, we proposed to work on the perception of localized distortion at a small temporal and spatial horizon. We present here the fundamental research questions and challenges in the domain with a focus on methods to collect perceptual judgments in subjective studies and metrics that can help us to derive an estimate of the perception of distortions by humans.

Perceptual annotation of local distortions in videos: tools and datasets

“…In future work, we plan to collect a large-scale dataset of tubecontents using this method, following the research proposal we presented at ACM MMSys'22 Doctoral Symposium in [13].…”

Predicting local distortions introduced by AV1 using Deep Features

“…In [11], a solution 3 is proposed to improve MLDS to select quadruplets for inter-content scaling efficiently. Intercontent scaling enables human observers to compare and rate the relative distortions introduced by codec across regions in a video frame [12]. The method has been compared with tripletbased and pairwise comparison subjective methodologies in [13] and validated for a crowdsourcing scenario in [14] where noisy annotations from outliers/spammers need to be handled.…”