Intra- and inter-observer variability in measurement of target lesions: implication on response evaluation according to RECIST 1.1

Abstract

BackgroundThe assessment of cancer treatment in oncological clinical trials is usually based on serial measurements of tumours’ size according to the Response Evaluation Criteria in Solid Tumours (RECIST) guidelines. The aim of our study was to evaluate the variability of measurements of target lesions by readers as well as the impact on response evaluation, workflow and reporting.Patients and methodsTwenty oncologic patients were included to the study with CT examinations from thorax to pelvis performed at a 64 slices CT scanner. Four readers defined and measured the size of target lesions independently at baseline and follow-up with PACS (Picture Archiving and Communication System) and LMS (Lesion Management Solutions, Median technologies, Valbonne Sophia Antipolis, France), according to the RECIST 1.1 criteria. Variability in measurements using PACS or LMS software was established with the Bland and Altman approach. The inter- and intra-observer variabilities were calculated for identical lesions and the overall response per case was determined. In addition, time required for evaluation and reporting in each case was recorded.ResultsFor single lesions, the median intra-observer variability ranged from 4.9–9.6% (mean 5.9%) and the median inter-observer variability from 4.3–11.4% (mean 7.1%), respecting different evaluation time points, image systems and observers. Nevertheless, the variability in change of Δ sum longest diameter (LD), mandatory for classification of the overall response, was 24%. The overall response evaluation assessed by a single respectively different observer was discrepant in 6.3% respectively 12% of the cases compared with the mean results of multiple observers. The mean case evaluation time was 286s vs. 228s at baseline and 267s vs. 196s at follow-up for PACS and LMS, respectively.ConclusionsUni-dimensional measurements of target lesions show low intra- and inter-observer variabilities, but the high variability in change of Δ sum LD shows the potential for misclassification of the overall response according to the RECIST 1.1 guidelines. Nevertheless, the reproducibility of RECIST reporting can be improved for the case assessment by a single observer and by mean results of multiple observers. Case-based evaluation time was shortened up to 27% using custom software.

Inter-observer reproducibility of semi-automatic tumor diameter measurement and volumetric analysis in patients with lung cancer

“…Using manual uni-dimensional measurements, the 95% limits of inter-observer agreement (−39.4% to 28.4%) were much higher. This is consistent with data reported for inter-observer variability in measurement of target lesions [27]. This variability is outside the range of clinical acceptability (<20%).…”

Intraobserver and interobserver variability in computed tomography size and attenuation measurements in patients with renal cell carcinoma receiving antiangiogenic therapy: Implications for alternative response criteria

“…It is known that interobserver variation in tumor measurements is greater than intraobserver variation, and multiple prior reports have advocated the use of a single observer or independent review committees to improve consistency in serial measurements in any 1 patient. [24][25][26][27][28][29] The results of the current study support the use of a single experienced radiologist for serial scan measurements to limit measurement variability when using a smaller change in tumor…”

“…The results of the current study demonstrate that a 10% tumor shrinkage threshold is within the 95% limits of agreement for interobserver variability, indicating that the 10% tumor shrinkage response threshold is within the range of measurement error using 2 radiologist observers. It is known that interobserver variation in tumor measurements is greater than intraobserver variation, and multiple prior reports have advocated the use of a single observer or independent review committees to improve consistency in serial measurements in any 1 patient . The results of the current study support the use of a single experienced radiologist for serial scan measurements to limit measurement variability when using a smaller change in tumor size to define response.…”

Re‐evaluating transarterial chemoembolization for the treatment of Hepatocellular Carcinoma: Consensus recommendations and review by an International Expert Panel

“…Although not compared directly to mRECIST, Muenzel et al . demonstrated low intra‐ and inter‐observer variabilities for measurements of single target lesions using RECIST, but the high variability in change in ∆ sum LD reveals the potential for misclassification of the overall response according to the RECIST guidelines . The authors suggest that reproducibility of RECIST reporting can be improved for the case assessment by a single reader and mean results of multiple readers.…”