Automatic Detection of Game Engine Artifacts Using Full Reference Image Quality Metrics

Abstract

Contemporary game engines offer an outstanding graphics quality but they are not free from visual artifacts. A typical example is aliasing, which, despite advanced antialiasing techniques, is still visible to the game players. Essential deteriorations are the shadow acne and peter panning responsible for deficiency of the shadow mapping technique. Also Z-fighting, caused by the incorrect order of drawing polygons, significantly affects the quality of the graphics and makes the gameplay more difficult. In this work, we propose a technique, in which visibility of deteriorations is uncovered by the objective image quality metrics (IQMs). We test the efficiency of a simple mathematically based metric and advanced IQMs: a Spatial extension of CIELAB (S-CIELAB), the Structural SIMilarity Index (SSIM), the Multiscale Structural SIMilarity Index (MS-SSIM), and the High Dynamic Range Visual Difference Predictor-2 (HDR-VDP-2). Additionally, we evaluate the Color Image Difference (CID) metric, which is recommended to detect the differences in colors. To find out which metric is the most effective for the detection of the game engine artifacts, we build a database of manually marked images with representative set of artifacts. We conduct subjective experiments in which people manually mark the visible local artifacts in the screenshots from the games. Then the detection maps averaged over a number of observers are compared with results generated by IQMs. The obtained results show that SSIM and MS-SSIM metrics outperform other techniques. However, the results are not indisputable, because, for small and scattered aliasing artifacts, HDR-VDP-2 metrics report the results most consistent with the average human observer. As a proof of concept, we propose an application in which resolution of the shadow maps is controlled by the SSIM metric to avoid perceptually visible aliasing artifacts on the shadow edges.

Dataset and Metrics for Predicting Local Visible Differences

“…The peter panning image contains distortion caused by shadow mapping where the shadow is detached from an object casting it [Piórkowski et al 2017]. The images also contain small differences in pixel values due to shading and post-processing effects in the game engine.…”

“…perceptionpatterns (34 images) from [Čadík et al 2013] are artificial patterns designed to expose well known perceptual phenomena, such as luminance masking, contrast masking and contrast sensitivity. Datasets aliasing (22 images), peterpanning (10 images), shadowacne (9 images), downsampling (27 images) and zfighting (10 images) are derived from [Piórkowski et al 2017] and contain real-time rendering artifacts. Those images were created using popular game engines (i.e.…”

Comparison of subjective methods, with and without explicit reference, for quality assessment of 3D graphics

“…In addition, it uses a continuous quality scale (0-100). Note that for graphics applications requiring localized information on the distortion visibility, the methodology based on the local marking of visible distortions is commonly used [Piórkowski et al 2017;Wolski et al 2018]. In such subjective experiments, observers manually mark the visible local artifacts in the impaired images.…”

Current Trends and Advances in Image Quality Assessment

“…Image quality assessment methods, particularly fullreference metrics which can be considered as similarity measures, may be useful for many different purposes. One of such "natural" areas of their applications may be the image and video quality assessment for computer games and computer graphics, as well as computer-generated images [76] with future possible applications in virtual and augmented reality.…”