The accuracy and robustness of a registration method depend on a number of factors, such as imaging modality, image content and image degrading effects, the class of spatial transformation used for registration, similarity measure, optimization, and numerous implementation details. The complex interdependence of these factors makes the assessment of the influence of a particular factor on registration difficult, although it is often desirable to have some estimate of such influences prior to registration. The similarity measure used to create the cost function is one of the factors that most influences the quality of registration. Traditionally, limited information on the behavior of a similarity measure is obtained either by studying the quality of the final registration or by drawing plots of similarity measure values obtained by translating or rotating one image relative to the "gold standard." In this paper, we present a protocol for a more thorough, optimization-independent, and systematic statistical evaluation of similarity measures. This protocol estimates a similarity measure's capture range, the number, location and extent of local optima, and the accuracy and distinctiveness of the global optimum. To show that the proposed evaluation protocol is viable, we have conducted several experiments with nine similarity measures and real computed tomography and magnetic resonance (MR) images of a spine phantom, MR brain images, and MR and positron emission tomography brain images, for which "gold standard" registrations were available. We have also studied the impact of histogram bin size on the behavior of nine similarity measures. The proposed evaluation protocol is useful for selecting the best similarity measure and corresponding optimization method for a particular application, as well as for studying the influence of sampling, interpolation, histogram bin size, partial image overlap, and image degradation, such as noise, intensity inhomogeneity, and geometrical distortions on the behavior of a similarity measure.
