Perceived quality assessment of polygonal meshes using observer studies: a new extended protocol

Abstract

The complexity of a polygonal mesh is usually reduced by applying a simplification method, resulting in a similar mesh having less vertices and faces. Although several such methods have been developed, only a few observer studies are reported comparing the perceived quality of the simplified meshes, and it is not yet clear how the choice of a given method, and the level of simplification achieved, influence the quality of the resulting mesh, as perceived by the final users. Similar issues occur regarding other mesh processing methods such as smoothing. Mesh quality indices are the obvious less costly alternative to user studies, but it is also not clear how they relate to perceived quality, and which indices best describe the users behavior. This paper describes on going work concerning the evaluation of perceived quality of polygonal meshes using observer studies, while looking for a quality index which estimates user performance. In particular, given some results obtained in previous studies, a new experimental protocol was designed and a study involving 55 users was carried out, which allowed their validation, as well as further insight regarding mesh quality, as perceived by human observers.

Assessing Visual Quality of 3-D Polygonal Models

“…The results of this work show that automatic measures of fidelity (e.g., Bolin's [29], Metro [5], mean squared error (MSE) approaches) are successful at predicting experimental ratings, less successful at predicting preferences, and largely unsuccessful at predicting naming times. On the other hand, when the task is based on comparing different models, ranking is stated to be better than rating the models because the given ratings do not necessarily reflect the perceptual distance between the compared models [35], [36]. The experimental measures used in several user studies can be found in Table I.…”

“…This can be achieved by animating the object or viewpoint as in [21], [35], as well as giving free viewpoint control to the user as in [8], [12], [36]. Furthermore, animations affect the perception of the models such that, in the study of Rogowitz and Rushmeier [35], artifacts caused by simplification are less visible when the objects are rotating rather than standing still.…”

“…Too few levels (compared cases) may not sufficiently reflect the tested operation, whereas a large number of levels may not be feasible, as they would require too many subjects. For simplification case, there are studies using three [30], [35] to seven [36] levels (including the originals) of simplification.…”

“…On the other hand, if the stimuli come from different types of distortions applied on very different 3D models, then it becomes much more difficult to correlate with human judgement. These data from Table III come from many different sources; for example, in the simplification column, we have synthesized results from Cleju and Saupe [42], Watson et al [30], and Lavoué [16] using a database from Silva et al [36]. Most of these results also come from the recent study of Lavoué and Corsini [8], who made a quantitative comparison of several 3D perceptual metrics according to several subjective databases.…”

“…For simple scenarios (a single mesh processing method or a single model), even simple metrics (e.g., Hausdorff) are [16], [30], [36], [42] Unif. noise addition [8], [19] Watermarking [11], [12], [14], [19] Masking scenario [14] Non-unif.…”