A methodology for {evaluating range image segmentation algorithms is proposed. This methodology involves 1) a common set of 40 laser range finder images and 40 structured light scanner images that have manually specified ground truth and 2) a set of defined performance metrics for instances of correctly segmented, missed, and noise regions, over-and undersegmentation, and accuracy of the recovered geometry. A tool is used to objectively-compare a machine generated segmentation against the specified ground truth. Four research groups have contributed to evaluate their own algorithm for segmenting a range image into planar patches. Index Terms-Experimental comparison of algorithms, range image segmentation, low level processing, performance evaluation In general, standardized segmentation error metrics are needed to kelp advance the state-of-the-art. No quantitative metrics are measured on standard test images in most of today's research environments.

Over-segmentation',`under-segmentation',`good results' and similar subjective terms appear frequently in the literature on range image segmentation. However, even though the need for standardized segmentation error metrics has been long recognized, no formal methodology for evaluating a range image segmentation has appeared. This paper describes a framework in which to carry out such an evaluation. With this framework, a more rigorous and objective comparison of range image segmentation techniques can be performed. We have developed many key issues, including a formal de nition of the range image segmentation problem, a comprehensive data set to use in evaluation, a method for creating ground truths, and a set of formally de ned metrics to classify segmentation results against ground truths.

No abstract