A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy

Mackay, K.; Bernstein, D.; Glocker, Ben; Kamnitsas, Konstantinos; Taylor, Alexandra

doi:10.1016/j.clon.2023.01.016

Cited by 29 publications

(11 citation statements)

References 128 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The migration distance is related to the Hausdorff distance 5,6 . The Hausdorff distance between three‐dimensional regions

X

and Y is defined as

\begin{equation*}{d_H}\;\left( {X,Y} \right) = \;{\mathrm{max}}\left( {\mathop {\max }\limits_{x \in X} \mathop {\min }\limits_{y \in Y} d\left( {x,y} \right),\;\mathop {\max }\limits_{y \in Y} \mathop {\min }\limits_{x \in X} d\left( {x,y} \right)} \right),\end{equation*}

where

d( {x,y} )

, as above, is the Euclidean distance between

x

and

y

.…”

Section: Methodsmentioning

confidence: 99%

“…The migration distance is related to the Hausdorff distance. 5,6 The Hausdorff distance between F I G U R E 1 Schematic illustrations of migration distance from the dark to light ROIs. In general, the migration distance is the smallest isotropic expansion of the dark ROI that encompasses the light ROI.…”

Section: Relationship To the Hausdorff Distancementioning

confidence: 99%

“…The radiotherapy process critically relies on region-ofinterest (ROI) geometric comparison metrics in a diverse range of applications including plan evaluation, 1,2 quality assurance, 3,4 and contouring uncertainty. 5,6 These metrics can be coarsely classified as volume-or distance-based and, while they are typically applied for contour comparisons, they are also beneficial for dosimetric comparisons as in the volume overlap of a specified dose level with a nearby organ-at-risk. The utility of these comparison metrics is often elevated when used in higher-level frameworks, such as the amalgamation of the dosimetric-difference and distance-toagreement in the gamma index for plan delivery quality assurance.…”

Section: Introductionmentioning

confidence: 99%

“…The radiotherapy process critically relies on region‐of‐interest (ROI) geometric comparison metrics in a diverse range of applications including plan evaluation, 1,2 quality assurance, 3,4 and contouring uncertainty 5,6 . These metrics can be coarsely classified as volume‐ or distance‐based and, while they are typically applied for contour comparisons, they are also beneficial for dosimetric comparisons as in the volume overlap of a specified dose level with a nearby organ‐at‐risk.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Technical note: The migration distance – a unidirectional distance metric for region‐of‐interest comparisons

Stewart,

Sahgal,

Hudson

et al. 2023

Medical Physics

View full text Add to dashboard Cite

BackgroundThe radiotherapy process relies on several metrics in determining a notion of “distance” from one three‐dimensional region‐of‐interest (ROI) to another. The majority are symmetric (or commutative) and do not contain information pertaining to directionality. Growth versus regression, for example, is not inherently distinguished by these metrics.PurposeThe purpose of this work was to formalize a unidirectional distance metric, motivated by radiotherapy margin concepts, which we term the migration distance. Informally, the migration distance from ROI X to Y is the minimum isotropic expansion of X such that Y is completely encompassed by the expansion. If Y is contained within X, the migration distance is negative with magnitude equal to the maximum isotropic contraction of X such that Y remains contained within contraction. The metric is demonstrated by quantifying glioblastoma interfraction target changes.MethodsAn explicit mathematical formulation of the migration distance is presented and contrasted with the related Hausdorff distance. The results are demonstrated for the gross tumor volume (GTV) dynamics of a glioblastoma cohort consisting of 111 patients that underwent standard chemoradiotherapy with offline MR imaging at planning, fraction 10, fraction 20, and 1‐month post radiotherapy.ResultsThe mean ± SD of the GTV migration distance relative to planning was 5.9 ± 3.9 mm at fraction 10, 6.2 ± 4.4 mm at fraction 20, and 7.9 ± 7.1 mm at 1‐month post radiotherapy. The maximum GTV migration distance across all patients at the same timepoints was 20.4, 20.7, and 45.5 mm, respectively.ConclusionsWe have proposed and demonstrated a unidirectional distance metric. The migration distance may have applications in the quantification of anatomical changes, planning target volume designs, and dosimetric radiotherapy plan assessment.

show abstract

“…The migration distance is related to the Hausdorff distance 5,6 . The Hausdorff distance between three‐dimensional regions

X

and Y is defined as

\begin{equation*}{d_H}\;\left( {X,Y} \right) = \;{\mathrm{max}}\left( {\mathop {\max }\limits_{x \in X} \mathop {\min }\limits_{y \in Y} d\left( {x,y} \right),\;\mathop {\max }\limits_{y \in Y} \mathop {\min }\limits_{x \in X} d\left( {x,y} \right)} \right),\end{equation*}

where

d( {x,y} )

, as above, is the Euclidean distance between

x

and

y

.…”

Section: Methodsmentioning

confidence: 99%

Section: Relationship To the Hausdorff Distancementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Technical note: The migration distance – a unidirectional distance metric for region‐of‐interest comparisons

Stewart,

Sahgal,

Hudson

et al. 2023

Medical Physics

View full text Add to dashboard Cite

show abstract

“…Notably, within several AI communities, there has been growing enthusiasm to shift from conventional “model-centric” AI approaches (ie, improving a model while keeping the data fixed), to “data-centric” AI approaches (ie, improving the data while keeping a model fixed). 4 Although balancing both approaches is typically ideal for crafting the optimal solution for specific-use cases, most research in RT auto-contouring has prioritized algorithmic modifications aimed at enhancing quantitative contouring performance based on geometric (ie, structural overlap) indices 5 —a clear testament to the “model-centric” AI paradigm.…”

Section: Introductionmentioning

confidence: 99%

Evolving Horizons in Radiation Therapy Auto-Contouring: Distilling Insights, Embracing Data-Centric Frameworks, and Moving Beyond Geometric Quantification

Wahid,

Cardenas,

Marquez

et al. 2024

Advances in Radiation Oncology

View full text Add to dashboard Cite

Radiomics as a measure superior to common similarity metrics for tumor segmentation performance evaluation

Akramova,

Watanabe

2024

J Applied Clin Med Phys

View full text Add to dashboard Cite

PurposeTo propose radiomics features as a superior measure for evaluating the segmentation ability of physicians and auto‐segmentation tools and to compare its performance with the most commonly used metrics: Dice similarity coefficient (DSC), surface Dice similarity coefficient (sDSC), and Hausdorff distance (HD).Materials/methodsThe data of 10 lung cancer patients’ CT images with nine tumor segmentations per tumor were downloaded from the RIDER (Reference Database to Evaluate Response) database. Radiomics features of 90 segmented tumors were extracted using the PyRadiomics program. The intraclass correlation coefficient (ICC) of radiomics features were used to evaluate the segmentation similarity and compare their performance with DSC, sDSC, and HD. We calculated one ICC per radiomics feature and per tumor for nine segmentations and 36 ICCs per radiomics feature for 36 pairs of nine segmentations. Meanwhile, there were 360 DSC, sDSC, and HD values calculated for 36 pairs for 10 tumors.ResultsThe ICC of radiomics features exhibited greater sensitivity to segmentation changes than DSC and sDSC. The ICCs of the wavelet‐LLL first order Maximum, wavelet‐LLL glcm MCC, wavelet‐LLL glcm Cluster Shade features ranged from 0.130 to 0.997, 0.033 to 0.978, and 0.160 to 0.998, respectively. On the other hand, all DSC and sDSC were larger than 0.778 and 0.700, respectively, while HD varied from 0 to 1.9 mm. The results indicated that the radiomics features could capture subtle variations in tumor segmentation characteristics, which could not be easily detected by DSC and sDSC.ConclusionsThis study demonstrates the superiority of radiomics features with ICC as a measure for evaluating a physician's tumor segmentation ability and the performance of auto‐segmentation tools. Radiomics features offer a more sensitive and comprehensive evaluation, providing valuable insights into tumor characteristics. Therefore, the new metrics can be used to evaluate new auto‐segmentation methods and enhance trainees' segmentation skills in medical training and education.

show abstract

A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy

Cited by 29 publications

References 128 publications

Technical note: The migration distance – a unidirectional distance metric for region‐of‐interest comparisons

Technical note: The migration distance – a unidirectional distance metric for region‐of‐interest comparisons

Evolving Horizons in Radiation Therapy Auto-Contouring: Distilling Insights, Embracing Data-Centric Frameworks, and Moving Beyond Geometric Quantification

Radiomics as a measure superior to common similarity metrics for tumor segmentation performance evaluation

Contact Info

Product

Resources

About