Predicting breast cancer types on and beyond molecular level in a multi-modal fashion

Abstract

Accurately determining the molecular subtypes of breast cancer is important for the prognosis of breast cancer patients and can guide treatment selection. In this study, we develop a deep learning-based model for predicting the molecular subtypes of breast cancer directly from the diagnostic mammography and ultrasound images. Multi-modal deep learning with intra- and inter-modality attention modules (MDL-IIA) is proposed to extract important relations between mammography and ultrasound for this task. MDL-IIA leads to the best diagnostic performance compared to other cohort models in predicting 4-category molecular subtypes with Matthews correlation coefficient (MCC) of 0.837 (95% confidence interval [CI]: 0.803, 0.870). The MDL-IIA model can also discriminate between Luminal and Non-Luminal disease with an area under the receiver operating characteristic curve of 0.929 (95% CI: 0.903, 0.951). These results significantly outperform clinicians’ predictions based on radiographic imaging. Beyond molecular-level test, based on gene-level ground truth, our method can bypass the inherent uncertainty from immunohistochemistry test. This work thus provides a noninvasive method to predict the molecular subtypes of breast cancer, potentially guiding treatment selection for breast cancer patients and providing decision support for clinicians.

iCluF: an unsupervised iterative cluster-fusion method for patient stratification using multiomics data

“…Therefore, these subtypes are clearly not classified as distinctive and independent tumor types ( Larsen et al 2013 ). This might be the reason why several computational studies primarily emphasized predicting only four BRCA subtypes (Basal, HER2, Luminal A, and Luminal B) instead of five ( Jaber et al 2020 , Phan et al 2021 , Zhang et al 2023 ). We also ran i CluF with K = 4 after removing the normal subtype from training, and we only predicted four clinical subtypes.…”

Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine

“…Radiomic features can provide a better overall representation of cancer, enabling more accurate classification and treatment. For example, previous studies have shown that combining MMG and ultrasound images to extract features has high accuracy in discriminating luminal and non-luminal diseases [ 93 ].…”

Progress in research on ultrasound radiomics for predicting the prognosis of breast cancer

“…Currently, CUS radiomics is mostly used to assist preoperative, noninvasive diagnosis of molecular subtypes of breast cancer. Research has mainly focused on the diagnosis of triple‐negative, luminal type, and human epidermal growth factor receptor‐2 positive breast cancer with an area under the curve (AUC) range of approximately 0.76–0.93 [ 17 , 18 , 19 , 20 , 21 ]. In addition, Jiang et al [ 22 ] have constructed a deep convolutional neural network based on 4828 CUS images from 1275 patients for diagnosing the four molecular subtypes of breast cancer.…”

“…The accuracy of this model is reportedly 80.07% (95% confidence interval [CI]: 76.49–83.23) to 97.02% (95% CI: 95.22–98.16) and 87.94% (95% CI: 85.08–90.31) to 98.83% (95% CI: 97.60–99.43) for the two test cohorts of each subtype. Furthermore, Zhang et al [ 21 ] combined CUS with mammography images from 3360 paired cases and proposed multimodal deep learning with intra‐ and intermodality attention modules to predict molecular subtypes of breast cancer. The accuracy of the model was 88.5% (95% CI: 86.0–90.9).…”