Evaluation of diagnostic accuracy in free-response detection-localization tasks using ROC tools

Bandos, Andriy I.; Obuchowski, Nancy A.

doi:10.1177/0962280218776683

Cited by 7 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, LMNB1 functions in the nuclear envelope lamina, possesses the transcriptional coregulatory activity, and has an important role in DNA replication, cell aging, and stress responses. Besides, it is positively correlated with tumor stage, tumor size, and number of nodules, therefore rendering itself a biomarker for early LIHC in tumor tissues and plasma [29]. In addition, SMARCA4 may enhance the growth and invasion of LIHC cells [30], while NRF1 is involved in stimulating the mitochondrial DNA (mtDNA) replication and transcription within LIHC [31], all of which are revealed in the network.…”

Section: Discussionmentioning

confidence: 99%

Development of an immune-related prognostic index associated with hepatocellular carcinoma

Yang

Sang

2020

Aging

View full text Add to dashboard Cite

Liver hepatocellular carcinoma (LIHC), an inflammation-associated cancer induced by a variety of etiological factors, is still one of the most prevalent and lethal cancers in human population. In this study, the expression profiles of immune-related genes (IRGs) were integrated with the overall survival (OS) of 378 LIHC patients based on the Cancer Genome Atlas (TCGA) dataset. Moreover, the differentially expressed and survival related IRGs among LIHC patients were predicted through the computational difference algorithm and COX regression analysis. As a result, 7 genes, including HSPA4, S100A10, FABP6, CACYBP, HDAC1, FCGR2B and SHC1, were retrieved to construct a predictive model associated with the overall survival (OS) of LIHC patients. Typically, the as-constructed model performed moderately in predicting prognosis, which was also correlated with tumor grade. Functional enrichment analysis revealed that the genes of high-risk group were actively involved in mRNA binding and the spliceosome pathway. Intriguingly, the prognostic index established based on IRGs reflected infiltration by multiple types of immunocytes. Our findings screen several IRGs with clinical significance, reveal the drivers of immune repertoire, and illustrate the importance of a personalized, IRG-based immune signature in LIHC recognition, surveillance, and prognosis prediction.

show abstract

Section: Discussionmentioning

confidence: 99%

Development of an immune-related prognostic index associated with hepatocellular carcinoma

Yang

Sang

2020

Aging

View full text Add to dashboard Cite

show abstract

“…The comparison between AI and mpMRI was made at PI-RADSv2 1 or greater representing all detected lesions and PI-RADSv2 3 or greater representing suspicious lesions. For lesion-based analysis, reader sensitivity and free-response ROC analysis was performed [15]. Reader statistics were averaged across all readers and by experience level.…”

Section: Discussionmentioning

confidence: 99%

Multicenter Multireader Evaluation of an Artificial Intelligence–Based Attention Mapping System for the Detection of Prostate Cancer With Multiparametric MRI

Mehralivand

Harmon

Shih

et al. 2020

American Journal of Roentgenology

View full text Add to dashboard Cite

One proposed solution is to use artificial intelligence (AI)-based detection systems.With the help of machine learning, classification algorithms can be trained to predict results and outcomes, provided that enough training data are available. In 2017, we at the National Cancer Institute [7] proposed an AI system based on intensity and texture analysis and a random forest classification algorithm. This system was validated in a large multireader multicenter study in 2018 [8]. Results of that study revealed an increase in detection of transition zone lesions among moderately experienced readers only. Overall, however, the AI system was equivalent to conventional MRI interpretation [8]. In that study, color-coded prediction maps were used to draw attention to AI-detected lesions. Feedback from the study suggested that prediction maps compromised the interaction between the radiologists and the AI system with resultant decreased accuracy for some readers. To address this issue a new AI detection system with more expert annotated

show abstract

“…Estimation and comparison of the predictive values were performed using ROC analysis on the augmented FROC data (iROI‐ROC) 37 and the generalized linear model (proc genmod, SAS v.9.4, SAS Institute, Cary NC) accounting for simultaneous detections 38 . Due to the lack of complete information of the validation data, the concordance between the detections of the compared methods was estimated by the concordance between the volumes that contain identified regions with sufficiently high scores.…”

Section: Methodsmentioning

confidence: 99%

Developing breast lesion detection algorithms for digital breast tomosynthesis: Leveraging false positive findings

2022

View full text Add to dashboard Cite

Background Due to the complex nature of digital breast tomosynthesis (DBT) in imaging techniques, reading times are longer than 2D mammograms. A robust computer‐aided diagnosis system in DBT could help radiologists reduce their workload and reading times. Purpose The purpose of this study was to develop algorithms for detecting biopsy‐proven breast lesions on DBT using multi‐depth level convolutional models and leveraging non‐biopsied samples. As biopsied positive samples in a lesion dataset are limited, we hypothesized that false positive (FP) findings by detection algorithms from non‐biopsied benign lesions could improve detection algorithms by using them as data augmentation. Approach We first extracted 2D slices from DBT volumes with biopsy‐proven breast lesions (cancer and benign), with non‐biopsied benign lesions (actionable), and for controls. Then, to provide lesion continuity along the z‐direction, we combined a lesion slice with its immediate adjacent slices to synthesize 2.5‐dimensional (2.5D) images of the lesion by assigning them into R, G, and B color channels. We used 224 biopsy‐proven lesions from 39 cancer and 62 benign patients from a DBTex challenge dataset of 1000 scans. We included the 2.5D images of immediate neighboring slices from the lesion's center to increase the number of training samples. For lesion detection, we used the YOLOv5 algorithm as our base network. We trained a baseline algorithm (medium‐depth level) using biopsied samples to detect actionable FPs in non‐biopsied images. Afterward, we fine‐tuned the baseline model on the augmented image set (actionable FPs added). For lesion inferencing, we processed the DBT volume slice‐by‐slice to estimate bounding boxes in each slice, and then combined them by connecting bounding boxes along the depth via volumetric morphological closing. We trained an additional model (large) with deeper‐depth levels by repeating the above process. Finally, we developed an ensemble algorithm by combining the medium and large detection models. We used the free‐response operating characteristic curve to evaluate our algorithms. We reported mean sensitivity per FPs per DBT volume only for biopsied views and sensitivity at 2‐false positives per image (2FPI) for all views. However, due to the limited accessibility to the truth of the challenge validation and test datasets, we used sensitivity at 2FPI for statistical evaluation. Results For the DBTex independent validation set, the medium baseline model achieved a mean sensitivity of 0.627 FPs per DBT volume, and a sensitivity of 0.640 at 2FPI. After adding actionable FP lesions, the model had an improved 2FPI of 0.769 over the baseline (p‐value = 0.013). Our ensemble algorithm with multi‐depth levels (medium + large) achieved a mean sensitivity of 0.815 FPs per DBT volume and an improved sensitivity at 2FPI of 0.80 over the baseline (p‐value < 0.001) on the validation set. Finally, our ensemble model achieved a mean sensitivity of 0.786 FPs per DBT volume and a sensitivity of 0.743 at 2FPI on the DB...

show abstract

Evaluation of diagnostic accuracy in free-response detection-localization tasks using ROC tools

Cited by 7 publications

References 31 publications

Development of an immune-related prognostic index associated with hepatocellular carcinoma

Development of an immune-related prognostic index associated with hepatocellular carcinoma

Multicenter Multireader Evaluation of an Artificial Intelligence–Based Attention Mapping System for the Detection of Prostate Cancer With Multiparametric MRI

Developing breast lesion detection algorithms for digital breast tomosynthesis: Leveraging false positive findings

Contact Info

Product

Resources

About