Weakly supervised detection and classification of basal cell carcinoma using graph-transformers on whole slide images

Abstract

The high incidence rates of basal cell carcinoma (BCC) cause a significant burden at pathology laboratories. The standard diagnostic process is time-consuming and prone to inter-pathologist variability. Despite the application of deep learning approaches in grading of other cancer types, there is limited literature on the application of vision transformers to BCC on whole slide images (WSIs). A total of 1831 WSIs from 479 BCCs, divided into training and validation (1434 WSIs from 369 BCCs) and testing (397 WSIs from 110 BCCs) sets, were weakly annotated into four aggressivity subtypes. We used a combination of a graph neural network and vision transformer to 1) detect the presence of tumor (two classes), 2) classify the tumor into low and high-risk subtypes (three classes), and 3) classify four aggressivity subtypes (five classes). Using an ensemble model comprised of the models from cross-validation, accuracies of 93.5%, 86.4%, and 72% were achieved on two, three, and five class classifications, respectively. These results show high accuracy in both tumor detection and grading of BCCs. The use of automated WSI analysis could increase workflow efficiency and possibly overcome inter-pathologist variability.

Generating clinical-grade pathology reports from gigapixel whole slide images with HistoGPT

“…Although current neural networks can also predict tumor thickness or tumor subtypes with good accuracy, as has been shown in particular for basal cell carcinoma [33][34][35][36][37][38][39] , they require a large amount of high-quality, precisely annotated data for training and are not flexible enough to be used for tasks other than those for which they were trained. That is, these models are fully supervised and do not operate in a zero-shot fashion.…”

A survey of Transformer applications for histopathological image analysis: New developments and future directions

“…For breast cancer histopathological image classification, DCET-Net [ 72 ] proposed a dual-stream convolution-expanded transformer architecture; Breast-Net [ 51 ] explores the ability of ensemble learning techniques using four Swin transformer architectures; HATNet [ 52 ] uses end-to-end vision transformers with a self-attention mechanism; ScoreNet [ 16 ] developed an efficient transformer-based architecture that integrates a coarse-grained global attention framework with a fine-grained local attention mechanism framework; LGVIT [ 73 ] built a local–global ViT model by introducing a new local–global MHSA mechanism and a ghost geed-forward network block into the network; dMIL-transformer [ 53 ] developed a two-stage double max–min multiple-instance learning (MIL) transformer architecture that combines both the spatial and morphological information of the cancer regions. Other than breast cancer classification, transformers have also been applied to other histopathological image cancer classification tasks, such as bone cancer classification (NRCA-FCFL [ 74 ]), brain cancer classification (ViT-WSI [ 17 ], ASI-DBNet [ 54 ], Ding et al [ 55 ]), colorectal cancer classification (MIST [ 75 ], DT-DSMIL [ 56 ]), gastric cancer classification (IMGL-VTNet [ 57 ]), kidney subtype classification (i-ViT [ 59 ], tRNAsformer [ 58 ]), thymoma or thymic carcinoma classification (MC-ViT [ 76 ]), lung cancer classification (GTP [ 46 ], FDTrans [ 60 ]), skin cancer classification (Wang et al [ 45 ]), and thyroid cancer classification (Wang et al [ 77 ], PyT2T-ViT [ 41 ], Wang et al [ 78 ]) using different transformer-based architectures. Furthermore, other transformer models such as Transmil [ 65 ], KAT [ 61 ], ViT-based unsupervised contrastive learning architecture [ 79 ], DecT [ 66 ], StoHisNet [ 80 ], CWC-transformer [ 63 ], LA-MIL [ 44 ], SETMIL [ 81 ], Prompt-MIL [ 67 ], GLAMIL [ 67 ], MaskHIT [ 82 ], HAG-MIL [ 68 ], MEGT [ 47 ], MSPT [ 70 ], and HistPathGPT [ 69 ] have also been evaluated on more than one tissue type, such as liver, prostate, breast, brain, gastric, kidney, lung, colorectal, and so on, for h...…”

“…Dwivedi et al [42] developed a graph transformer network (GTN) that supports the use of specific domain information as edge features and provides interpretability via self-attention modules that locate the key regions of the graphs for prediction. AMIGO [43], LA-MIL [44], Wang et al [45], and GTP [46], MEGT [47] are some examples of graph-based transformer models that have been proposed for different histopathological image classification tasks.…”

“…Figure 9 shows some examples of SOTA transformer architectures developed for histopathological image classification. DT-DSMIL [56]), gastric cancer classification (IMGL-VTNet [57]), kidney subtype classification (i-ViT [59], tRNAsformer [58]), thymoma or thymic carcinoma classification (MC-ViT [76]), lung cancer classification (GTP [46], FDTrans [60]), skin cancer classification (Wang et al [45]), and thyroid cancer classification (Wang et al [77], PyT2T-ViT [41], Wang et al [78]) using different transformer-based architectures. Furthermore, other transformer models such as Transmil [65], KAT [61], ViT-based unsupervised contrastive learning architecture [79], DecT [66], StoHisNet [80], CWC-transformer [63], LA-MIL [44], SETMIL [81], Prompt-MIL [67], GLAMIL [67], MaskHIT [82], HAG-MIL [68], MEGT [47], MSPT [70], and HistPathGPT [69] have also been evaluated on more than one tissue type, such as liver, prostate, breast, brain, gastric, kidney, lung, colorectal, and so on, for histopathological image classification using different transformer approaches.…”

Self-supervised artificial intelligence predicts recurrence, metastasis and disease specific death from primary cutaneous squamous cell carcinoma at diagnosis

“…However, light microscopy has a limited role in determining prognosis itself, given potential for sampling error as well as inherent inter-reader variability in histopathologic features. Recently, supervised machine learning algorithms have been used in the cutaneous melanoma realm to identify prognostically important features, response to immunotherapy, one-year disease-free survival and mutation prediction with promising results (21)(22)(23)(24). Current studies looking at machine learning for KC, such as squamous cell carcinoma are, however, limited.…”