A review of methods of analysis in contouring studies for radiation oncology

Abstract

Inter-observer variability in anatomical contouring is the biggest contributor to uncertainty in radiation treatment planning. Contouring studies are frequently performed to investigate the differences between multiple contours on common datasets. There is, however, no widely accepted method for contour comparisons. The purpose of this study is to review the literature on contouring studies in the context of radiation oncology, with particular consideration of the contouring comparison methods they employ. A literature search, not limited by date, was conducted using Medline and Google Scholar with key words: contour, variation, delineation, inter/intra observer, uncertainty and trial dummy-run. This review includes a description of the contouring processes and contour comparison metrics used. The use of different processes and metrics according to tumour site and other factors were also investigated with limitations described. A total of 69 relevant studies were identified. The most common tumour sites were prostate (26), lung (10), head and neck cancers (8) and breast (7).The most common metric of comparison was volume used 59 times, followed by dimension and shape used 36 times, and centre of volume used 19 times. Of all 69 publications, 67 used a combination of metrics and two used only one metric for comparison. No clear relationships between tumour site or any other factors that may influence the contouring process and the metrics used to compare contours were observed from the literature. Further studies are needed to assess the advantages and disadvantages of each metric in various situations.

A phantom assessment of achievable contouring concordance across multiple treatment planning systems

“…The JI [18][19][20], DICE [4], Hausdorff distance [21] and Kappa (κ) statistic [22,23] outlined in Supplementary Table 1, are metrics commonly used to establish inter-observer variation [6]. JI and DICE values from CERR were verified in 3D Slicer [24][25][26] and MILXview and were consistent to within 2 significant figures.…”

The utility of atlas-assisted segmentation in the male pelvis is dependent on the interobserver agreement of the structures segmented

“…[11][12][13] As recommended by these reviews, we used three different types of metrics to evaluate the outlining agreement achieved. The simplest was using a comparison of volume [in terms of percentage volume difference (PVD)].…”

Impact of contouring variability on dose-volume metrics used in treatment plan optimization of prostate IMRT

“…The DSC metric is commonly used in studies of contouring variability [10]. In this study, DSC demonstrated the highest compliance value for bladder (0.971), followed by prostate (0.838), and then rectum ( 0.771) There are several possible reasons for the high similarity among bladder contours compared to other contours.…”