Crafting the MPEG metrics for objective and perceptual quality assessment of volumetric videos

Abstract

Efficient objective and perceptual metrics are valuable tools to evaluate the visual impact of compression artifacts on the visual quality of volumetric videos (VVs). In this paper, we present some of the MPEG group efforts to create, benchmark and calibrate objective quality assessment metrics for volumetric videos represented as textured meshes. We created a challenging dataset of 176 volumetric videos impaired with various distortions and conducted a subjective experiment to gather human opinions (more than 5896 subjective scores were collected). We adapted two state-of-the-art model-based metrics for point cloud evaluation to our context of textured mesh evaluation by selecting efficient sampling methods. We also present a new image-based metric for the evaluation of such VVs whose purpose is to reduce the cumbersome computation times inherent to the point-based metrics due to their use of multiple kd-tree searches. Each metric presented above is calibrated (i.e., selection of best values for parameters such as the number of views or grid sampling density) and evaluated on our new ground-truth subjective dataset. For each metric, the optimal selection and combination of features is determined by logistic regression through cross-validation. This performance analysis, combined with MPEG experts’ requirements, lead to the validation of two selected metrics and recommendations on the features of most importance through learned feature weights.

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

“…Even though such a metric would more than likely be application‐dependent, we believe that exploring such a direction of research would be valuable for advancing our understanding of VH animation realism. In addition to the studies already mentioned in this survey, such as the one of Cabrero‐Daniel et al [CDMH*21] where authors proposed a quality metric of pedestrians trajectories in a crowd, we can mention here other recent studies from other fields of research tending in this direction, such as recent objective quality assessment metrics for volumetric videos represented as textured meshes [NDD*23, MNGL23]. We can also refer to the realism metric for generated LiDAR point clouds proposed by Triess et al [TRPZ22] or the deep learning‐based perceptual metric for trees modeling introduced by Polasek et al [PHBČ21].…”

Geometric Accuracy Analysis Between Neural Radiance Fields (Nerfs) and Terrestrial Laser Scanning (Tls)

“…Thus, a 3D-to-2D projection-based mechanism (Yang et al, 2020) enables a simplified objective comparison between point cloud quality evaluation and conventional image-based measurements. Consequently, with aim to build an open standard for compactly representing 3D point clouds, the Moving Picture Experts Group (MPEG) proposes establishing quality metrics, such as point-to-point (p2point), point-to-plane (p2plane) and point-to-mesh (p2mesh) (Marvie et al, 2023). The p2point metric quantifies the distances between corresponding points to measure the degree of distortion, p2plane projects the obtained p2point distances along the surface normal direction, while p2mesh reconstructs the surface and then measures the distance from a point to the surface, but the efficiency is strongly dependent on the accuracy of the surface reconstruction algorithm (Liu et al, 2023).…”