Subjective and Objective Quality Assessment of the SoftCast Video Transmission Scheme

Abstract

SoftCast-based linear video coding and transmission (LVCT) schemes have been proposed as a promising alternative to traditional video coding and transmission schemes in wireless environments. Currently, the performance of LVCT schemes is evaluated by means of traditional objective scores such as PSNR or SSIM. Nevertheless, since the compression is performed in a very different way from traditional coding schemes such as HEVC, visual artifacts are also quite different and deserve to be subjectively assessed. In this paper, we propose a subjective quality assessment of SoftCast, pioneer and standard of the LVCT schemes. This study aims to better understand the tradeoffs between the LVCT parameters that can be tuned to improve the quality. These parameters, including different GoP-sizes, Compression Ratios (CR) and Channel Signal-to-Noise Ratio (CSNR), are used to generate a dataset of 85 videos. A Double Stimulus Impairment Scale (DSIS) test is performed on the received videos to assess the perceived quality. Results show that the key characteristic of SoftCast, the linear relation between CSNR and PSNR, is also observed with the Mean-Opinion Scores (MOS), except at high CSNR where the quality saturates. In addition, Bjøntegaard model is used to quantify the trade-offs between CR, GoP-size and CSNR, depending on the intended application. Finally, the performance of objective metrics compared to the obtained MOS is evaluated. Results show that Multi-Scale SSIM (MS-SSIM), SSIM and Video Multimethod Assessment Fusion (VMAF) metrics offer the best correlation with the MOS values.

Performance Assessment of the Adaptive GoP-size extension of the Wireless SoftCast Video Scheme

“…4. In addition to the previous used metrics, we add the MS-SSIM [14] and VMAF [15] ones as suggested in [9] due to their strong correlation with the subjective scores. The VMAF metric ranges between [0-100] whereas the (MS)-SSIM metrics range between [0-1].…”

“…Each frame is split into 64 chunks as in [1], [7], [12]. As classically done in the literature [7], [9], [12], only the luminance is considered hereafter.…”

“…3) or intraframe cuts, i.e., a shot change inside the same scene (Fig. 4), • Two additional objective metrics: the Multi Scale-SSIM (MS-SSIM) as well as the Video Multi-method Assessment Fusion (VMAF) metrics due to their high correlation with human judgements in a SoftCast context [9].…”

Soft Delivery: Survey on A New Paradigm for Wireless and Mobile Multimedia Streaming

“…Although the transmitted samples are distorted according to their SNR, the receiver regards the received samples as scaled DCT coefficients. This process avoids all cliff, leveling, and staircase effects because the sender does not need to estimate the channel condition, and the noise level in the received samples faithfully reflects the instantaneous channel condition [23]. Consequently, pseudoanalog modulation ensures that the received video quality is proportional to the instantaneous channel quality, as shown in Fig.…”

A Perceptual Study of the Decoding Process of the SoftCast Wireless Video Broadcast Scheme

“…First, the estimators are characterized mathematically. Then, an objective evaluation of the received quality offered by the two estimators is performed by considering the metrics suggested by [10] as they offer the best correlation with human judgment. Finally, a subjective evaluation based on a forced-choice PairWise Comparison (PWC) and statistical analysis is proposed.…”