Model for defining and reporting reference-based validation protocols in medical image processing

Abstract

Jannin P, Grova C, Maurer C. Model for designing and reporting reference based validation procedures in medical image processing. Int Journ. Comput. Assisted Radiol and Surg. 2006 1(2) reference that is assumed to be very close or equal to the correct solution. In this paper, we propose a model for defining and reporting reference-based validation protocols in medical image processing.Materials and Methods. The model was built using an ontological approach. Its components were identified from the analysis of initial publications (mainly reviews) on medical image processing, especially registration and segmentation, and from discussions with experts from the medical imaging community during international conferences and workshops. The model was validated by its instantiation for 38 selected papers that include a validation study, mainly for medical image registration and segmentation.Results. The model includes the main components of a validation procedure and their interrelationships. A checklist for reporting reference-based validation studies for medical image processing was also developed.Conclusion. The proposed model and associated checklist may be used in formal reference-based validation studies of registration and segmentation and for the complete and accurate reporting of such studies. The model facilitates the standardization of validation terminology and methodology, improves the comparison of validation studies and results, provides insight into the validation process, and, finally, may lead to better a quality management and decision making.

Electromagnetic Tracking in Medicine—A Review of Technology, Validation, and Applications

“…At some point, experiments should be transferred into an actual clinical environment [63], [137,Chap. 18].…”

“…For example, if the jitter is significantly larger than the static distortion, calibration will not be of much help. The same is true if the patient's image is of low quality, the registration is performed poorly, or the instruments are not calibrated properly [63]. At the end of the day, the clinicians using the EM tracking systems will be the ones to affirm the usability of the tool, with accuracy being a critical component, but also soft requirements other than pure target numbers may be important.…”

Comparative Validation of Single-Shot Optical Techniques for Laparoscopic 3-D Surface Reconstruction

“…To characterize the strengths and weaknesses of each technique, we investigated reconstruction accuracy in a comprehensive in vitro study, involving different organs with various shape and texture and also tested reconstruction robustness with respect to the pose of the endoscope and the amount of blood or smoke present in the scene. To ensure reproducibility of our results, we applied a previously published model for defining and reporting reference-based validation protocols in medical image computing [14].…”

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

“…Careful validation, both in controlled environments as well as in clinical scenarios, is crucial for establishing a new system in the clinic (Jannin et al, 2002(Jannin et al, , 2006. So far, most of the validation studies relating to 3D surface reconstruction have been performed on numerically simulated data (Hu et al, 2007;, on phantom models with known ground truth geometry and motion characteristics (Hu et al, 2007;Wu and Qu, 2007;Richa et al, 2008b;Noonan et al, 2009) or with ground truth data obtained by scanning techniques Hu et al, 2007;Wu and Qu, 2007;Richa et al, 2008b;Noonan et al, 2009;Stoyanov et al, 2010).…”

“…In order to allow reporting validation experiments in a standardized manner, Jannin et al proposed a model for defining and reporting reference-based validation protocols in medical image processing. (Jannin et al, 2006). The model and an associated checklist facilitate the standardization of validation terminology and methodology and thus the comparison of validation studies.…”