Weakly Supervised Breast Cancer Classification on WSI Using Transformer and Graph Attention Network

Abstract

Recently, multiple instance learning (MIL) has been successfully used in weakly supervised breast cancer classification on whole‐slide imaging (WSI) and has become an important assistance for breast cancer diagnosis. However, existing MIL methods have limitations in considering the global contextual information of pathological images. Additionally, their ability to handle spatial relationships among instances should also be improved. Therefore, inspired by transformer and graph deep learning, this study proposes a novel classification method of WSI breast cancer pathological images based on BiFormer and graph attention network (BIMIL‐GAT). In the first stage of instance selection, BiFormer utilizes the two‐stage self‐attention computation mechanism from coarse‐grained region to fine‐grained region to strengthen the global feature extraction ability, which can obtain accurate pivotal instances. Simultaneously, the aim of the second stage is to effectively strengthen the spatial correlation between instances through GAT, thereby improving the accuracy of bag‐level prediction. The experimental results show that BIMIL‐GAT achieves the area under curve (AUC) value of 95.92% on the Cameylon‐16 dataset, which outperforms the baseline model by 3.36%. In addition, our method also shows strong competitiveness in the MSK external extended dataset, which further proves its effectiveness and advancement.