Measuring the similarity of target volume delineations independent of the number of observers

Abstract

The variability of target delineations is a topic of interest in radiotherapy. The similarity of delineations is often quantified by use of a conformity index (CI) defined as the ratio of common to encompassing volume. Several forms of CI are in use, but no consensus exists on how to calculate the CI for more than two delineations. This study proposes a generalization of the CI applicable to any number of delineations. The generalization of the CI was developed, unbiased with respect to the number of delineations. Numerical values were calculated for clinical and theoretical cases, and differences with other forms of CI were considered. A simple expression could be derived, applicable to any number of delineations, and is equivalent to the known CI for two delineations. The use of this index is advised, although another frequently used index obtained from averaging the CI between all possible pairs of delineations results in minor differences. The use of the third generalization for the CI which is based upon the volume common to all delineations shows a clear dependence upon the number of delineations and is discouraged.

Interobserver variation among pathologists for delineation of tumor on H&E-sections of laryngeal and hypopharyngeal carcinoma. How good is the gold standard?

“…After delineation, the sections, including the tumor outlines, were scanned at 300 dpi resolution. For separate digitization of the lines drawn by the pathologist an in-house developed software package [10], used in clinical radiotherapy practice, was applied to manually trace the tumor outline by a researcher. The digitized lines were then projected on scanned H&E-sections without delineations.…”

Interobserver variation in parotid gland delineation: a study of its impact on intensity-modulated radiotherapy solutions with a systematic review of the literature

“…The Jaccard coefficient is defined as the ratio of the volume of intersection or overlap to the encompassing volume for any two parotid contours (Figure 1). The conformity index was then calculated for each patient by averaging the Jaccard coefficient between all possible pairs of contours for that patient [18]. The conformity index could therefore range between zero (no overlap between contours) and one (identical contours), and was determined separately for radiation oncologists and radiologists.…”

A phantom assessment of achievable contouring concordance across multiple treatment planning systems

“…As such there has been an increasing trend to assess, and reduce, the variability of these target volumes. This study determined the highest concordance metrics achievable, and how these metrics (details given in Supplementary Table 1) including; Jaccard Index (JI also known as conformity index or concordance index (CI) [6,12]), CI pairs the average of all possible pairs of the JI (equates to CI gen when mutual variability between all observers is the same [13]), Dice Coefficient (DICE or DSC), Volume Overlap Index (VOI), the generalised kappa statistic and Hausdorff Distance (HD), may vary in a best case phantom scenario considering: multiple sites, variation between TPSs, shapes, volume, tools utilized and adherence to auto-threshold settings within the protocol.…”