External Validation of Diagnostic Models to Estimate the Risk of Malignancy in Adnexal Masses

Abstract

Purpose: To externally validate and compare the performance of previously published diagnostic models developed to predict malignancy in adnexal masses.Experimental Design: We externally validated the diagnostic performance of 11 models developed by the International Ovarian Tumor Analysis (IOTA) group and 12 other (non-IOTA) models on 997 prospectively collected patients. The non-IOTA models included the original risk of malignancy index (RMI), three modified versions of the RMI, six logistic regression models, and two artificial neural networks. The ability of the models to discriminate between benign and malignant adnexal masses was expressed as the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and likelihood ratios (LR þ , LR À ). Results: Seven hundred and forty-two (74%) benign and 255 (26%) malignant masses were included. The IOTA models did better than the non-IOTA models (AUCs between 0.941 and 0.956 vs. 0.839 and 0.928). The difference in AUC between the best IOTA and the best non-IOTA model was 0.028 [95% confidence interval (CI), 0.011-0.044]. The AUC of the RMI was 0.911 (difference with the best IOTA model, 0.044; 95% CI, 0.024-0.064). The superior performance of the IOTA models was most pronounced in premenopausal patients but was also observed in postmenopausal patients. IOTA models were better able to detect stage I ovarian cancer.Conclusion: External validation shows that the IOTA models outperform other models, including the current reference test RMI, for discriminating between benign and malignant adnexal masses. Clin Cancer Res; 18(3); 815-25. Ó2011 AACR.

Strategies to diagnose ovarian cancer: new evidence from phase 3 of the multicentre international IOTA study

“…A third limitation is that most of the patients in the IOTA phase 3 study were scanned by the same experienced examiners as in the centres where the IOTA methods were developed, or by examiners that had already adopted the IOTA examination technique and terminology. This may explain why the results of IOTA phase 3 confirm those of previous IOTA studies that showed excellent test performance of all IOTA strategies (Timmerman et al , 2005, 2007, 2008, 2010a, 2010b; Van Holsbeke et al , 2009, 2012; Ameye et al , 2012; Kaijser et al , 2013; Sayasneh et al , 2013a, 2013b). On the other hand, validation studies of LR1, LR2 and SRs performed outside IOTA studies reported similar results (Hartman et al , 2012; Nunes et al , 2012, 2013; Alcázar et al , 2013), and there is now evidence that the IOTA strategies retain their performance in the hands of sonographers and relatively inexperienced doctors (Hartman et al , 2012; Nunes et al , 2012, 2013; Alcázar et al , 2013; Sayasneh et al , 2013a, 2013b).…”

Diagnosis, Treatment, and Follow-Up of Borderline Ovarian Tumors

“…However, the performance of RMI is poor when applied to masses in young women and pathology that is difficult to be characterised with ultrasound; also, the sensitivity of the test is low. In a study by van Holsbeke et al, the RMI missed 73% of BOTs (31/42 cases) [106].…”

Triaging women with ovarian masses for surgery: observational diagnostic study to compare RCOG guidelines with an International Ovarian Tumour Analysis (IOTA) group protocol

“…We used the data from phase 2 of the IOTA study, which have also been used in previous publications. [17][18][19][20] Patients were recruited between November 2005 and October 2007 in 11 oncology referral centres, three referral centres for ultrasonography, and five general hospitals in eight countries (Belgium, Sweden, Italy, UK, Czech Republic, Poland, China and Canada). The centres are listed in Table 1.…”

“…These models then underwent temporal validation (IOTA phase 1b, n = 507) and external validation (IOTA phase 2, n = 1938) with excellent performance, 16,17 also in comparison with RMI and other models from the literature as evaluated using the area under the receiver operating characteristics curve. 18 On external validation in 12 centres LR2 and RMI had areas under the curve of 0.95 and 0.91, respectively, and the advantage of LR2 over RMI in terms of area under the curve was larger for premenopausal women than for postmenopausal women. 18 The aim of this study is to compare the RCOG protocol with a protocol based on the IOTA logistic regression model LR2 with respect to the classification of adnexal masses as being at low, moderate or high risk of malignancy.…”

“…18 On external validation in 12 centres LR2 and RMI had areas under the curve of 0.95 and 0.91, respectively, and the advantage of LR2 over RMI in terms of area under the curve was larger for premenopausal women than for postmenopausal women. 18 The aim of this study is to compare the RCOG protocol with a protocol based on the IOTA logistic regression model LR2 with respect to the classification of adnexal masses as being at low, moderate or high risk of malignancy. The key objective is to investigate whether the IOTA protocol improves the selection of women referred for potentially radical surgery by labelling more benign tumours as low risk, more invasive tumours as high risk, and classifying fewer tumours as moderate risk.…”