Evaluation framework of superpixel methods with a global regularity measure

Abstract

In the superpixel literature, the comparison of state-of-the-art methods can be biased by the non-robustness of some metrics to decomposition aspects, such as the superpixel scale. Moreover, most recent decomposition methods allow to set a shape regularity parameter, which can have a substantial impact on the measured performances. In this paper, we introduce an evaluation framework, that aims to unify the comparison process of superpixel methods. We investigate the limitations of existing metrics, and propose to evaluate each of the three core decomposition aspects: color homogeneity, respect of image objects and shape regularity. To measure the regularity aspect, we propose a new global regularity measure (GR), which addresses the non-robustness of state-of-the-art metrics. We evaluate recent superpixel methods with these criteria, at several superpixel scales and regularity levels. The proposed framework reduces the bias in the comparison process of state-of-the-art superpixel methods. Finally, we demonstrate that the proposed GR measure is correlated with the performances of various applications.

Robust superpixels using color and contour features along linear path

“…Note that recent works, e.g., Giraud et al (2017c); Stutz et al (2017) show the high correlation between the undersegmentation error (Neubert and Protzel, 2012) and the ASA metric (10). Therefore, the ASA measure is sufficient to evaluate the respect of image objects.…”

Texture-Aware Superpixel Segmentation

“…Hence, the barycenter is encouraged to move to a homogeneous textured area and to be contained within the superpixel (see Figure 2(d)). This also increases the shape regularity, which is a desirable property [24].…”

“…TASP is compared to the recent state-of-the-art methods SLIC [14], ERGC [15], ETPS [20], LSC [18], SNIC [16], and SCALP [19]. Performances are evaluated with standard Achievable Segmentation Accuracy (ASA), and contour detection metric F-measure (F) as defined in [24], and we report quantitative results for an average number of 250 superpixels. Initial image LSC [18] SNIC [16] SCALP [19] TASP Fig.…”

Multi-scale superpatch matching using dual superpixel descriptors

“…Several works have used superpixels in non-local frameworks, e.g., [12,29], or in unsupervised learning-based superpixel matching approaches using random forests [6,16]. Nevertheless, the geometrical irregularity of such decompo-sitions [11] (i.e., in terms of shape, adjacency or contour smoothness) can become an issue, since neighborhood information is crucial to compute accurate matches in terms of context.…”