A hierarchical self‐attention‐guided deep learning framework to predict breast cancer response to chemotherapy using pre‑treatment tumor biopsies

Abstract

BackgroundPathological complete response (pCR) to neoadjuvant chemotherapy (NAC) has demonstrated a strong correlation to improved survival in breast cancer (BC) patients. However, pCR rates to NAC are less than 30%, depending on the BC subtype. Early prediction of NAC response would facilitate therapeutic modifications for individual patients, potentially improving overall treatment outcomes and patient survival.PurposeThis study, for the first time, proposes a hierarchical self‐attention‐guided deep learning framework to predict NAC response in breast cancer patients using digital histopathological images of pre‐treatment biopsy specimens.MethodsDigitized hematoxylin and eosin‐stained slides of BC core needle biopsies were obtained from 207 patients treated with NAC, followed by surgery. The response to NAC for each patient was determined using the standard clinical and pathological criteria after surgery. The digital pathology images were processed through the proposed hierarchical framework consisting of patch‐level and tumor‐level processing modules followed by a patient‐level response prediction component. A combination of convolutional layers and transformer self‐attention blocks were utilized in the patch‐level processing architecture to generate optimized feature maps. The feature maps were analyzed through two vision transformer architectures adapted for the tumor‐level processing and the patient‐level response prediction components. The feature map sequences for these transformer architectures were defined based on the patch positions within the tumor beds and the bed positions within the biopsy slide, respectively. A five‐fold cross‐validation at the patient level was applied on the training set (144 patients with 9430 annotated tumor beds and 1,559,784 patches) to train the models and optimize the hyperparameters. An unseen independent test set (63 patients with 3574 annotated tumor beds and 173,637 patches) was used to evaluate the framework.ResultsThe obtained results on the test set showed an AUC of 0.89 and an F1‐score of 90% for predicting pCR to NAC a priori by the proposed hierarchical framework. Similar frameworks with the patch‐level, patch‐level + tumor‐level, and patch‐level + patient‐level processing components resulted in AUCs of 0.79, 0.81, and 0.84 and F1‐scores of 86%, 87%, and 89%, respectively.ConclusionsThe results demonstrate a high potential of the proposed hierarchical deep‐learning methodology for analyzing digital pathology images of pre‐treatment tumor biopsies to predict the pathological response of breast cancer to NAC.

Apriori prediction of chemotherapy response in locally advanced breast cancer patients using CT imaging and deep learning: transformer versus transfer learning

“…Saednia et al. ( 31 ) trained a hierarchical self-attention deep learning network to predict the response of NAC to LABC using digital histopathological images.…”

“…Likewise, in another study, outcome of radiotherapy for brain metastasis was predicted using the combination of deep learning features and clinical features. In this study, a deep convolutional neural network (CNN) was trained on MRI images to extract MRI features and thus deep textural MR-features are combined with clinical features to predict the outcome of treatment ( 31 ). Fujima et al.…”

Artificial intelligence’s impact on breast cancer pathology: a literature review

“…The study emphasizes the broad applicability of computational biomarkers, especially in routine clinical practice with H&E staining, and highlights considerations for improved model applicability and TIL scoring interpretation. Another study by Saednia et al [ 23 ] introduces a hierarchical deep learning framework for predicting breast cancer response to NAC using digital histopathological images. Unlike the previous study, this model incorporates a patch-level processing module, a tumor-level processing module, and a patient-level response prediction module, providing a comprehensive approach.…”