Inter-observer variability of manual contour delineation of structures in CT

Abstract

• This study quantifies the inter-observer variability of manual delineation of lesions and organ contours in CT. • The variability of manual delineations between two observers can be significant. Two and even three observers capture only a fraction of the full range of inter-observer variability observed in common practice. • Inter-observer manual delineation variability is necessary to establish a reference standard for radiologist training and evaluation and for the evaluation of automatic segmentation algorithms.

Radiomics feature reproducibility under inter-rater variability in segmentations of CT images

“…While all these works aim at assessing feature reproducibility with respect to segmentation variability, the segmentations based on which the analyses were carried out originate from only two raters at most. However, it has been shown recently by Joskowicz et al that inter-rater variability in segmentation of lesions in CT images cannot be adequately captured by two raters only 20 . In fact, it was found that more than three raters are required in order to capture the full distribution of plausible segmentations.…”

Texture analysis of computed tomography data using morphologic and metabolic delineation of esophageal cancer—relation to tumor type and neoadjuvant therapy response

“…Use of texture analysis requires segmentation of the volumes of interest to be used for processing. Earlier studies have applied manual segmentation where the tumor is manually delineated on all image slices where it is visible, which is a time-consuming process and suffers from high intra-and interobserver variability (64)(65)(66). Advances in computing power have enabled the development of several methods for automating parts of that procedure.…”

Automated Tortuosity Analysis of Nerve Fibers in Corneal Confocal Microscopy

“…It is worth noticing that inter-observer variability (IOV) is ubiquitous at many aspects of medical image analysis, such as the manual contour delineation of structures in CT [56], and fibre tortuosity estimation in the proposed task. With the rapid boost of deep learning-based medical imaging analysis, the IOV affects the development of deep learning-based methods from the training of network models to the evaluation of their performance [57].…”

“…With the rapid boost of deep learning-based medical imaging analysis, the IOV affects the development of deep learning-based methods from the training of network models to the evaluation of their performance [57]. As indicated in [56], the IOV in manual delineations for different structures in CT images is large and two or three observers may not be sufficient to establish the full range of IOV. Therefore, the conventional consensus amongst multiple observers is adopted in this paper.…”