Democratized image analytics by visual programming through integration of deep models and small-scale machine learning

Godec, Primož; Pančur, Matjaž; Ilenič, Nejc; Čopar, Andrej; Stražar, Martin; Erjavec, Aleš; Pretnar, Ajda; Demšar, Janez; Starič, Anže; Toplak, Marko; Žagar, Lan; Hartman, Jan; Wang, Hamilton; Bellazzi, Riccardo; Petrovič, Uroš; Garagna, Silvia; Zuccotti, Maurizio; Park, Dongsu; Shaulsky, Gad; Zupan, Blaž

doi:10.1038/s41467-019-12397-x

Cited by 69 publications

(56 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this study, we proposed a collaborative framework that enables integration of clinicians' prior knowledges and deep transferable image feature representations into an interpretable PI-Risk tool to improve the predictions of Gleason score. This integrated approach to data analysis can be generalized under the 'task-free image embedding with privileged deep networks' paradigm described by Zupan et al [23]. This study contributes important methodology accompanied with model interpretability to address a critical clinical question for PCa risk strati cation.…”

Section: Discussionmentioning

confidence: 98%

Integration of Clinical Identifications With Deep Transferrable Imaging Feature Representations Can Help Predict Prostate Cancer Aggressiveness and Outcome

Bao

Hou

Zhi

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: A pretreatment risk stratification of prostate cancer (PCa) is important for determining appropriate treatment option and prognostic prediction. Although numerous quantifiable approaches can provide fundamental insights in PCa, a clinical tool should leverage the integration of all data representations to enables detailed assessment. We developed a generalizable machine learning platform, designated PI-Risk, which incorporates clinicians’ prior identifications with deep transferrable imaging feature representations into predictive models for PCa Gleason grade. Methods: A retrospective study included 1442 biopsy-naïve patients from two tertiary care medical centers between January 2014 and December 2019. Model performance was typically evaluated against a “ground truth” with imaging-histopathologic annotations using receiver operating characteristic (ROC). Detection rates such as true positive, true negative, false positive and false negative rate were reported using a confusion matrix analysis. The Cox model’s performance was evaluated based on Harrell’s concordance index (C-index), calibration curves and Kaplan–Meier survival analysis. Results: In multinomial regression analyzing model, predicted IntraT-Rad G0 (odds ratio [OR], 3.01; 95% confidence intervals [CIs], 2.74–3.34, p < 0.001) and PI-RADS score 2 (OR, 2.88; 95% CIs, 2.47–3.17, p = 0.002) were two independent predictors of G0 stage. Predicted IntraT-Rad G1 (OR, 2.42; 95% CIs, 2.13–2.86, p < 0.001) and PSA 4-10 ng/ml (OR, 1.29; 95% CIs, 1.06–1.43, p = 0.037) were two independent predictors of G1 stage. PSA 10-20 ng/ml (OR, 1.52; 95% CIs, 1.32–1.67, p = 0.005), Predicted PeriT-DLR-SqueezeNet G1 (OR, 1.31; 95% CIs, 1.14–1.52, p = 0.009) and Predicted IntraT-Rad G3 (OR, 1.29; 95% CIs, 1.07–1.44, p = 0.011) were three independent predictors of G2 stage. PI-RADS score 5 (OR, 1.84; 95% CIs, 1.74–2.31, p < 0.001) were the independent predictor of G3 stage. PI-RADS score 5 (OR, 2.84; 95% CIs, 2.62–3.17, p < 0.001) and PSA > 100 ng/ml (OR, 1.69; 95% CIs, 1.36–1.83, p = 0.007) were two independent predictors of G4 stage. And the multivariate Cox analysis model shows that, among the 5 pretreatment risk factors (age, PSA, PCa location, PI-RADS score and PI-Risk score), PSA ≥ 20 ng/ml (OR, 1.58; 95% CI, 1.20-2.08; p = 0.001) and PI-Risk ≥ G3 (OR, 1.45; 95% CI, 1.12-1.88; p = 0.005) were the independent predictors of BCR.Conclusions: We concluded that the PI-Risk can offer a noninvasive alternative tool to stratify PCa aggressiveness. This enables a step towards PCa risk stratification.

show abstract

Section: Discussionmentioning

confidence: 98%

Integration of Clinical Identifications With Deep Transferrable Imaging Feature Representations Can Help Predict Prostate Cancer Aggressiveness and Outcome

Bao

Hou

Zhi

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For training in machine learning, we use the data science toolbox called Orange ( http://orange.biolab.si ) [ 6 , 8 , 9 ]. Orange is an open-source, cross-platform data mining and machine learning suite.…”

Section: Approachmentioning

confidence: 99%

“…Orange ( http://orange.biolab.si ) [ 6 ] is a visual programming environment that combines data visualization and machine learning. In the past years, we have been tailoring Orange towards a tool for education (e.g., [ 7 , 8 ]). We have used it to design short, practical hands-on workshops.…”

Section: Introductionmentioning

confidence: 99%

Hands-on training about overfitting

Demšar

Zupan

2021

PLoS Comput Biol

Self Cite

View full text Add to dashboard Cite

Overfitting is one of the critical problems in developing models by machine learning. With machine learning becoming an essential technology in computational biology, we must include training about overfitting in all courses that introduce this technology to students and practitioners. We here propose a hands-on training for overfitting that is suitable for introductory level courses and can be carried out on its own or embedded within any data science course. We use workflow-based design of machine learning pipelines, experimentation-based teaching, and hands-on approach that focuses on concepts rather than underlying mathematics. We here detail the data analysis workflows we use in training and motivate them from the viewpoint of teaching goals. Our proposed approach relies on Orange, an open-source data science toolbox that combines data visualization and machine learning, and that is tailored for education in machine learning and explorative data analysis.

show abstract

“…PeriT-DLR features were directly measured on MRI data using an image embedding toolbox (https://github.com/biolab) through ve pre-trained deep neural networks, i.e. DeepLoc, Inception v3, SqueenzeNet, VGG-16 and VGG-19 as embedders [23]. In order to obtain the representative imaging features of the target lesion, we used hand-cropped VOI as an attention to gate each embedder for analyzing PeriT-DLRs (i.e., regions around the PCa) in the center slice of an MRI scan.…”

Section: Development Performance and Validation Of Predictive Modelsmentioning

confidence: 99%

Integration of Clinical Identifications With Deep Transferrable Imaging Feature Representations Can Help Predict Prostate Cancer Aggressiveness and Outcome

Bao

Hou

Zhi

et al. 2021

Preprint

View full text Add to dashboard Cite

Objective: To develope a generalizable machine learning platform, designated PI-Risk, which incorporates clinicians’ prior identifications with deep transferrable imaging feature representations into predictive models for PCa Gleason grade. Patients and Methods: A retrospective study included 1442 biopsy-naïve patients from two tertiary care medical centers between January 2014 and December 2019. We investigated an interpretable risk assessment model (PI-Risk) to predict risk stratification of PCa using . The performance of PI-Risk model was independently tested on 232 internal test datasets, and on 539 external validation datasets. Model performance was typically evaluated against a “ground truth” with imaging-histopathologic annotations using receiver operating characteristic (ROC). Detection rates such as true positive, true negative, false positive and false negative rate were reported using a confusion matrix analysis. The Cox model’s performance was evaluated based on Harrell’s concordance index (C-index), calibration curves and Kaplan–Meier survival analysis. Results: The PI-Risk integrated with 10 risk factors is formed to accurate risk stratification. In multinomial regression analyzing model, predicted IntraT-Rad G0 and PI-RADS score 2 were two independent predictors of G0 stage. Predicted IntraT-Rad G1 (OR, 2.42; 95% CIs, 2.13–2.86, p < 0.001) and PSA 4-10 ng/ml were two independent predictors of G1 stage. PSA 10-20 ng/ml, Predicted PeriT-DLR-SqueezeNet G1 and Predicted IntraT-Rad G3 were three independent predictors of G2 stage. PI-RADS score 5 were the independent predictor of G3 stage. PI-RADS score 5 and PSA > 100 ng/ml were two independent predictors of G4 stage. Combined use of PSA 4-10 ng/ml, PI-RADS 1-2 and stacked IntraT-Rad G0-1 resulted in excellent NPV (94.1%) for CIS diseases; and combined use of PSA >100 ng/ml and PI-RADS 5 resulted in high PPV (79.8%) for high-risk PCa. In follow-up, patients stratified by PI-Risk showed significantly different biochemical recurrence rate after surgery.Conclusions: We concluded that the PI-Risk can offer a noninvasive alternative tool to stratify PCa aggressiveness. This enables a step towards PCa risk stratification.

show abstract

Democratized image analytics by visual programming through integration of deep models and small-scale machine learning

Cited by 69 publications

References 19 publications

Integration of Clinical Identifications With Deep Transferrable Imaging Feature Representations Can Help Predict Prostate Cancer Aggressiveness and Outcome

Integration of Clinical Identifications With Deep Transferrable Imaging Feature Representations Can Help Predict Prostate Cancer Aggressiveness and Outcome

Hands-on training about overfitting

Integration of Clinical Identifications With Deep Transferrable Imaging Feature Representations Can Help Predict Prostate Cancer Aggressiveness and Outcome

Contact Info

Product

Resources

About