A modified Wald interval for the area under the ROC curve (AUC) in diagnostic case-control studies

Kottas, Martina; Kuß, Oliver; Zapf, Antonia

doi:10.1186/1471-2288-14-26

Cited by 52 publications

(34 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ROC analysis is widely used in various fields, such as bioinformatics, medical statistics, radiology, pattern recognition, and machine learning [42-44]. In addition, some indicators from the ROC curve, such as the AUC, have been used as evaluation and construction of classifiers [45, 46]. Our results indicated that the AUC values of K trans were 0.790 and 0.816 at NRT and after the 5th RT, respectively.…”

Section: Discussionmentioning

confidence: 84%

Quantitative Multiparametric MRI May Augment the Response to Radiotherapy in Mid-Treatment Assessment of Patients with Esophageal Carcinoma

Xie

Pang³

et al. 2019

Oncol Res Treat

View full text Add to dashboard Cite

Objective: The purpose of this study was to assess the mid-treatment response to radiotherapy (RT) using dynamic contrast-enhanced MRI (DCE-MRI) and diffusion-weighted imaging (DWI) in patients with esophageal cancer (EC). Methods: 42 patients with squamous EC were prepared for DCE-MRI and DWI scans both before treatment (NRT) and after the fifth radiotherapy (5th RT). The patients were classified into two groups (complete response [CR] and partial response [PR]) according to tumor regression after treatment. The quantitative parameters of DCE-MRI (K^trans, Kep, Ve, and ADC) were measured. A receiver operating characteristic curve (ROC) was used to detect the efficiency of the above parameters. Results: After 1 month of RT, 29 patients were classified as CR and 11 patients were classified as PR. In the NRT group, the p values of K^trans, Kep, Ve, and ADC were 0.004, 0.078, 0.0008, and <0.0001, respectively. After the 5th RT, the p values of the above parameters were <0.001, 0.005, 0.108, and 0.365, respectively. In the NRT group, the areas under the ROC curves of K^trans, Ve, and ADC were 0.790, 0.617, and 0.737; the sensitivity values were 89.3, 92.5, and 90.0%; the specificity values were 69.4, 27.5, and 50.0%. In the 5th RT group, the areas under the ROC curves of K^trans and Kep were 0.816 and 0.804; the sensitivity values were 71.2 and 95.0%; the specificity values were 81.6 and 50.0%. Conclusion: DCE-MRI combined with DWI is effective in the early prediction of radiotherapeutic response of EC after the 5th RT other than after the traditional final treatment.

show abstract

Section: Discussionmentioning

confidence: 84%

Quantitative Multiparametric MRI May Augment the Response to Radiotherapy in Mid-Treatment Assessment of Patients with Esophageal Carcinoma

Xie

Pang³

et al. 2019

Oncol Res Treat

View full text Add to dashboard Cite

show abstract

“…Because the Wald interval depends on sample size, the method used for calculating the 95% CI of the AUC was a Wald model with correction for continuity. 7,8 The study unit for generating the AUC and 95% CI was the "patient." For that reason, we used the average of variables for each patient.…”

Section: Discussionmentioning

confidence: 99%

“…Cellular parameter values are expressed as means ± standard deviations (in μm); and, for ROC curve analysis, values are expressed as the area under the curve (AUC) with 95% confidence interval (CI). Because the Wald interval depends on sample size, the method used for calculating the 95% CI of the AUC was a Wald model with correction for continuity . The study unit for generating the AUC and 95% CI was the “patient.” For that reason, we used the average of variables for each patient.…”

Section: Methodsmentioning

confidence: 99%

Digital holographic microscopy as screening tool for cervical cancer preliminary study

Benzerdjeb

Garbar

Camparo

et al. 2016

Cancer Cytopathology

View full text Add to dashboard Cite

BACKGROUND:The liquid-based cytology Papanicolaou (Pap) test has gradually replaced the conventional Pap test, as it seems to be more efficient, in part because it uses computer assistance. However, Pap test diagnoses are still subjective and are cost-consuming or time-consuming. The authors describe a new digital holographic microscopy instrument and accompanying software. This technology provides an instantaneous, 3-dimensional image reconstruction of cells directly from the uterine cervical sample vial. METHODS: Analyses were performed using the proprietary digital holographic microscopy (DHM) instrument, computer, and software. Residual materials from 3 randomly chosen laboratories were analyzed and subjected to DHM, and the sensitivity and specificity of DHM were calculated for the detection neoplasia. RESULTS:In 41 specimens that yielded normal Pap test results, 1333 cells were analyzed using DHM; and, in 30 specimens that yielded abnormal Pap test results, 494 cells were analyzed using DHM. Available histologic correlation was as follows:4 histologic samples were diagnosed as grade 1 cervical intraepithelial neoplasia (corresponding to 2 cytologic diagnoses of low-grade squamous intraepithelial lesion and 2 cytologic diagnoses of atypical squamous cells of undetermined significance), 4 histologic samples were diagnosed as grade 3 cervical intraepithelial neoplasia (corresponding to 2 cytologic diagnoses of low-grade squamous intraepithelial lesion and 2 cytologic diagnosis of atypical squamous cells, cannot rule out high-grade squamous intraepithelial lesion), and 11 lesions were diagnosed as benign. Receiver operating characteristic curve analysis indicated that the best criteria were the nuclear-to-cytoplasmic ratio (area under the curve, 0.88; 95% confidence interval, 0.68-1.00) and the greatest nuclear dimension (area under the curve, 0.85; 95% confidence interval, 0.66-1.00). CONCLUSIONS: This preliminary study demonstrated for the first time that the DHM technique is suitable for the processing of gynecologic cervical samples. Nevertheless, DHM criteria and parameters could be better defined.Hopefully, holographic analysis will be performed automatically and will provide an instantaneous, cost-effective diagnosis from a closed vial with the preservation of all cellular material. Cancer Cytopathol 2016;124:573-80. V C 2016 American Cancer Society.

show abstract

“…Areas under receiver operating characteristic curves (AUCs) were computed for each finding with 95% confidence intervals computed by the exact Clopper-Pearson method. 25 In addition, we also estimated specificity and sensitivity that yield the highest harmonic mean with the 95% confidence intervals. We used the metrics module in the Python scikit-learn package version 0.18.2 (http://www.python.org) in the computation of the AUC and confusion matrix for sensitivity and specificity calculation.…”

Section: Evaluation Of the Algorithmsmentioning

confidence: 99%

Development and Validation of Deep Learning Models for Screening Multiple Abnormal Findings in Retinal Fundus Images

Son¹,

et al. 2020

View full text Add to dashboard Cite

Purpose: To develop and evaluate deep learning models that screen multiple abnormal findings in retinal fundus images. Design: Cross-sectional study. Participants: For the development and testing of deep learning models, 309 786 readings from 103 262 images were used. Two additional external datasets (the Indian Diabetic Retinopathy Image Dataset and e-ophtha) were used for testing. A third external dataset (Messidor) was used for comparison of the models with human experts. Methods: Macula-centered retinal fundus images from the Seoul National University Bundang Hospital Retina Image Archive, obtained at the health screening center and ophthalmology outpatient clinic at Seoul National University Bundang Hospital, were assessed for 12 major findings (hemorrhage, hard exudate, cottonwool patch, drusen, membrane, macular hole, myelinated nerve fiber, chorioretinal atrophy or scar, any vascular abnormality, retinal nerve fiber layer defect, glaucomatous disc change, and nonglaucomatous disc change) with their regional information using deep learning algorithms. Main Outcome Measures: Area under the receiver operating characteristic curve and sensitivity and specificity of the deep learning algorithms at the highest harmonic mean were evaluated and compared with the performance of retina specialists, and visualization of the lesions was qualitatively analyzed. Results: Areas under the receiver operating characteristic curves for all findings were high at 96.2% to 99.9% when tested in the in-house dataset. Lesion heatmaps highlight salient regions effectively in various findings. Areas under the receiver operating characteristic curves for diabetic retinopathy-related findings tested in the Indian Diabetic Retinopathy Image Dataset and e-ophtha dataset were 94.7% to 98.0%. The model demonstrated a performance that rivaled that of human experts, especially in the detection of hemorrhage, hard exudate, membrane, macular hole, myelinated nerve fiber, and glaucomatous disc change. Conclusions: Our deep learning algorithms with region guidance showed reliable performance for detection of multiple findings in macula-centered retinal fundus images. These interpretable, as well as reliable, classification outputs open the possibility for clinical use as an automated screening system for retinal fundus images.

show abstract

A modified Wald interval for the area under the ROC curve (AUC) in diagnostic case-control studies

Cited by 52 publications

References 27 publications

Quantitative Multiparametric MRI May Augment the Response to Radiotherapy in Mid-Treatment Assessment of Patients with Esophageal Carcinoma

Quantitative Multiparametric MRI May Augment the Response to Radiotherapy in Mid-Treatment Assessment of Patients with Esophageal Carcinoma

Digital holographic microscopy as screening tool for cervical cancer preliminary study

Development and Validation of Deep Learning Models for Screening Multiple Abnormal Findings in Retinal Fundus Images

Contact Info

Product

Resources

About