Contrastive self-supervised learning from 100 million medical images with optional supervision

Ghesu, Florin C.; Georgescu, Bogdan; Mansoor, Awais; Yoo, Youngjin; Neumann, Dominik; Patel, Pragneshkumar; Vishwanath, R. S.; Balter, James M.; Cao, Yue; Grbić, Saša; Comanicìu, Dorin

doi:10.1117/1.jmi.9.6.064503

Cited by 29 publications

(11 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, we demonstrated that our foundation model trained using self-supervised learning, provided robust quantitative biomarkers for predicting anatomical site, malignancy, and prognosis across three different use cases in four cohorts. Several studies [19][20][21] have demonstrated the efficacy of self-supervised learning in medicine where only limited data might be available for training deep learning networks. Our findings complement and extend this for identifying reliable imaging biomarkers for cancer-associated use cases.…”

Section: Discussionmentioning

confidence: 99%

Foundation Models for Quantitative Biomarker Discovery in Cancer Imaging

Pai,

Bontempi,

Hadzic

et al. 2023

Preprint

View full text Add to dashboard Cite

Foundation models represent a recent paradigm shift in deep learning, where a single large-scale model trained on vast amounts of data can serve as the foundation for various downstream tasks. Foundation models are generally trained using self-supervised learning and excel in reducing the demand for training samples in downstream applications. This is especially important in medicine, where large labeled datasets are often scarce. Here, we developed a foundation model for imaging biomarker discovery by training a convolutional encoder through self-supervised learning using a comprehensive dataset of 11,467 radiographic lesions. The foundation model was evaluated in distinct and clinically relevant applications of imaging-based biomarkers. We found that they facilitated better and more efficient learning of imaging biomarkers and yielded task-specific models that significantly outperformed their conventional supervised counterparts on downstream tasks. The performance gain was most prominent when training dataset sizes were very limited. Furthermore, foundation models were more stable to input and inter-reader variations and showed stronger associations with underlying biology. Our results demonstrate the tremendous potential of foundation models in discovering novel imaging biomarkers that may extend to other clinical use cases and can accelerate the widespread translation of imaging biomarkers into clinical settings.

show abstract

Section: Discussionmentioning

confidence: 99%

Foundation Models for Quantitative Biomarker Discovery in Cancer Imaging

Pai,

Bontempi,

Hadzic

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…For our task, we had annotated data and formulated pretraining and a supervision task. However, if more data without annotations are available, selfsupervised methods deserve exploration [31].…”

Section: A CCC Detectionmentioning

confidence: 99%

Deep Learning Based Detection of Collateral Circulation in Coronary Angiographies

Hatfaludi,

Bunescu,

Ciuşdel

et al. 2023

2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS)

View full text Add to dashboard Cite

Coronary artery disease (CAD) is the dominant cause of death and hospitalization across the globe.Atherosclerosis, an inflammatory condition that gradually narrows arteries and has potentially fatal effects, is the most frequent cause of CAD. Nonetheless, the circulation regularly adapts in the presence of atherosclerosis, through the formation of collateral arteries, resulting in significant longterm health benefits. Therefore, timely detection of coronary collateral circulation (CCC) is crucial for CAD personalized medicine. We propose a novel deep learning based method to detect CCC in angiographic images. Our method relies on a convolutional backbone to extract spatial features from each frame of an angiography sequence. The features are then concatenated, and subsequently processed by another convolutional layer that processes embeddings temporally. Due to scarcity of data, we also experiment with pretraining the backbone on coronary artery segmentation, which improves the results consistently. Moreover, we experiment with few-shot learning to further improve performance, given our low data regime. We present our results together with subgroup analyses based on Rentrop grading, collateral flow, and collateral grading, which provide valuable insights into model performance. Overall, the proposed method shows promising results in detecting CCC, and can be further extended to perform landmark based CCC detection and CCC quantification.

show abstract

“…Due to the higher precision of the input data, the 7/15 inbuilt PyTorch transforms are not directly used and must be re-implemented to fit the data format. Augmentations such as random intensity scaling and horizontal flipping are used along with a custom normalization of the DICOM images which includes histogram equalization and dynamic window scaling 27 .…”

Section: Implementation Detailsmentioning

confidence: 99%

AUCReshaping: Improved sensitivity at high-specificity

Bhat,

Mansoor,

Georgescu

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

The performance of deep-learning (DL) systems are generally measured using the Area under the Receiver-Operating-Curve (AU-ROC); however, the holistic nature of this metric fails to account for the performance, at a specific range(s) of sensitivity and specificity, at which system is intended to operate. Therefore, it is seen that two systems with the same AU-ROC perform very differently in the field. The issue is especially exaggerated for anomaly detection tasks: a very popular application of DL systems in various areas of research spanning medical imaging, industrial automation, manufacturing, cyber security, fraud detection, drug research, etc. This is primarily due to the fact that the datasets used to train these systems are heavily imbalanced, and the abnormality class usually has a highly skewed misclassification cost compared to the normal class. Traditionally, DL systems account for this by weighting the cost function or designing for different operating points on the ROC curve. This approach achieves reasonable results in most instances but, the system does not actively attempt to maximize performance for the chosen operating point. In this paper, we propose a novel function called AUCReshaping which reshapes the ROC curve only in the desired region by optimizing the sensitivity at a desired specificity. The reshaping is implemented through an adaptive and iterative boosting manner that allows the network to focus on relevant samples during learning. We mainly studied the impact of AUCReshaping on an abnormality detection Chest X-Ray (CXR) task, followed by a breast mammogram and credit card fraud detection task. The results demonstrate an increase of 2-40% in sensitivity at high-specificity, for a binary classification task.

show abstract

Contrastive self-supervised learning from 100 million medical images with optional supervision

Cited by 29 publications

References 45 publications

Foundation Models for Quantitative Biomarker Discovery in Cancer Imaging

Foundation Models for Quantitative Biomarker Discovery in Cancer Imaging

Deep Learning Based Detection of Collateral Circulation in Coronary Angiographies

AUCReshaping: Improved sensitivity at high-specificity

Contact Info

Product

Resources

About